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ABSTRACT 

This report describes the ccnceftualjlzation ^ 
development^ and evaluation^ of a handtcok (CI 016 ^32) fci project - 
directors responsible for iaplementing careej education pxoducts. 
Divided , into five chapters'^ this docuoeiit certains a chronological 
review of development- activities, including the use of tie prototype 
version in schools. Chapter 1 explains the ceed for the hiindbook^ the 
need for pioqrammatic research findings* «aiid the specif icat:ions ^or 
the handbook* In chapter 2 both the formative and i^umiativ^ ^ . 
engineering in the development of the tandbcck is reviewed^ Chapter 3 
describes the evaluation activities^ and chapter. U descrltes, the 
product utilization stage which included program mctiigring'j 
development of a technical ^plan; conduct of market anaily^is; 
development of production and packaging specif icaticn^; development 
of prcmotionr distribntionr and service specifications; preparation 
of a product utilization plan; and conduct c£ pxoducticn, promotion, - 
distribution, and service activities, linally, in chapter 5 the 
findings are summarized and claims and recciienda'tions for the 
handbook are presented. Appended materials comprise half cf this 
document, including a list of persons ccntlributing to the development 
of the handbook, summative evaluation materials, development' 
materials, and formative evaluation settings and results. (BH) 
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THE CENTEtl MISSION STA^TEMENT 



The Cente!r fpr Vbcatipnal Education's missi^bn is^^^^^^^t^^ ; 
increase the ability of diverse agencies, institii-, 
tions , ' and\organiz^rions to solve educationkl , ' 
problems relating ^to indivicaual career planning aind 
prepaLration/ Th^ Center fulfills its mission bV: V 

V: ■ , r- ' : 

• Generating knowledge^ through reseatch V - 

■ ^ 'Devel9t>ing educati(3)nal programs aiid products . 
/ Evaluating individual program needs and outcomes 

• ^Iftstalling educational programs 'anp prb^cts 

• Opea^'ating information systems and services 



Conducting « leadership .4e\felopment and training 
prqgraps . i , > . • 
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FOREWORD 



This report des 
and /^yaduation of a 
tors.' It contains d 
ties including the u 
authors are indebted 
directors: (1) thos 
participating^ in^the 
•This latter group wa 
directors from the s 
Appendix A. 



cribes the concept^rali , devei^opm^-iit ^ 

handbook for career - education «.pr6iject direc- 
chronologd^al review of developmeijt activi^.- 
sd of the prototype version in/schopls. -The 

to two groups of careet education , pro j^ct 
e using the prototype version and /,{2) tl^se, 
sununative evaluation of. the final product!^ 
s randomly seleqted from lists of project 
tajjfces. , TheJ-project directors are listed in 



Appreciation is ejc^cub^ to Alan.K^hler , Professor of Agri- 
cultural 'Educ^.t:ion at Iowa State University and Director of an' , 
Exemplary Pro3ectV in Career Education, for his assistance in -wri- 
ting spme? of the test situations^. We wish to acknowledge the 
contributions of ratters of'thQ test results. These pefsdns ajre^ 
Mary B« IJi^vit, formerly Chairperson of the Department of ^Voca-^ • 
Vtional-Technical Education in the Graduate School of Education at 
the Rutgers University; .Clarence Kron, Dean of the Sc^hool of 1 
Education at the University of Texas at Odessa; and Charlek RyanV 
Professor, Department of ^.Guidance and Counseling, University of 
Mainei. Respectively, they provided the technical expertise in; 
innova.tion^dif fusion , educati9r>al administration, .and career ^ 
education so necessary for evaluating the quality of ^impjementa'-^^ 
tion strategies. .\ ' , — ^ 

' . n : ■ . ' " . •■ ■, ■ 

The developjnent of a handbook for .iinf)lementing career educa- 
tion product^ represents t^e combined efforts of the sponsor, 
the Eduqfiitioj;! and Work Group at the National Institute of. Educa- 
tion^ theNdeyelopers, project stsaff at the Center for yocati9nal'^ 
Education; and prof essionals in the^rield. We believe tha't the V 
c<^tents of 'the handbook will contribute to effective and effi;- 
ci^nt use of . career education m^rt^rtal^ by students and teachers. 
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CHAPTER I 
[ijl^^^DpCTION . 



The Need f or . th^e Handbook, 



The Ca^reer Educati9n Xmplementation Handbook wa^ ^developed 
in response to a pe/cvasive n^dd in American education'. This - 
need was assQcia.t:ed with €h^. r6^ instruction to real / 

^lif-e goals of Students. Thevt^^ 

not/smooth foi^ 'm^ny- individuails • \ Hi^h dropout;* rates were occur- . • 
-i-ing in American education arid, mjjch floundering" from' job to job 
^ap . taking'. ^pj.ace^ as indiyiduals/seSxched for* the .best occupation 
' to/ meet/ th(^'r*> 'iftdly^idual* goals ; 7 

. ThisC loss or ^oellence in ' the public'- school, system was rec-^ 
ognized by government' officials sucJi ai^ Sidney P'. oMaxl'and, Jr. 
.^nd Qthers .who represent resporisible, leadership in American ^u- 
cation. . Such recognitions led toja^majofiL Investment in the 
development of currioiilum materials and other i^ristructional aids ' 
which would foster cajgjjj^r guid^ance pf students- in schools . jCar^r- 
educati9n models in the home/ •in bii^iaess/ 'atnd in schools were > 
developed. Concurrent ,with ., this iijvestment of resources oh th^' 

' national scene, states were beginning to mandate career edlication'^^ 
priorities. Instructional materials .yere develd^ped in isolated ' 
school systems and through coordinated approabKes under the ,lead- 

-extship" of state education departments ^ The movement toyrard career 
education 'Uad gained suff icieint moi^entum by 1974 that the U.S. 
Commissioner' of Education, T. H. Bell (1974), estimated that 

•almost 5,000 of' the 17,^00 school districts in the United Staters j 
and at least six state legislatures ^had enacted .career edification « 
legislation. Personnel, in the form 6f s£ate^ career education 

\coordinators^ w^re located .in at least three -fourths of the state 

^education departments. • The education amendments o£ 1974' (PL 93- 
380) contained a definitive statement'.,,^ career education. 

Grass roots support fop^careej: education as a .vehicle for 

facilitating improvement in the' educational systems receiyed^ • 
attention and support fjrom many sources. The American Vocational 
Association published a ^yearbook on career education^ (Magisos, - 
1973).* The National School ^ub-il^ic Relations Associatioi::^ (l974) ^ 
synthtesized trends and policies in a publication wl^ich reviewed 
the need for career education anci* identified some of the c^rtti- 
cisms voiced toward this .hiovement. This statement also Provided' 
a synthesis of some of ,t;^e state programs of career educ^iation. * 

' ■ - : ^ /• ■ . • ■ 
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A ♦more^ specif ic icomment pny state actiyit;S.lfes • irf career edacatioii ^ 
- proy:Saeca by the Council^ o^/ Chief State School Officers in a 

report autiaorfed Joy Jesser (1974) .^ A/ri^tiona'): assessment ' of pto-; 
>^anfs related to occupational educationT Projectc'^B.as^ fojand' 

neces'sary to issue a publication bn^caree^ * 
, bV-Wbrtzhlfigtort ./1974j . Following k ser^i^s of mini conference^ ^ 
conducted by thp USOE Office ofc^Caree^VEducation a pQlicy paper, 
.h^s been f ormulfated : which^ specif ies rSefin^^ions , ^assumptions/ and 
^ tasks associated with career^ education, yThis pqlicy paper, and a 
series of^mcmographs (the first one by Hp rr and Gramer, 1975) are 
being disseminated- to state education a^ncies and local? educa- . 
tion agencies to aid. them in- the implementation process. /• 

• *^ The magnitude/of this ; (Career education movement has resulted, 
in^any books and ^related, riia'terials being written. A book by 
Goldhaminefr and Taylor (1974)- jdescr^ibed *some af the parameters of 
• the career. ed^ca;ti_pn mpvement*. One of garland/ s most recent books 
' (197.41 dpscribe^ both the processes™ of *cmreer education reform 
and ^examples of programs .- The i;ntensity of intereS.t>in career 
ed'uc.ation progtams was indicated by :a publixration I5y ,Ray^n (1973) 
which indicate^ various soxxt^B of fundsi for carjeer t educ^t'iori pro- 
' grams. /Most recently , a H&ndjpook- of career education materials 
^ha^ been dev^lopad by the .Edt/catiohal Products Infojmaiiion Ex- 
jqha^ge^ under contract with 'bh^ National In&titute o'f Education^ 

. The availjability of such a ^multitude .of materials in this 
V -fi-eld acce^ats 'the need fqr an iitiplementaMzion system. In addition 
.to ^ information searcH^ and retprieval capacities, ^state education 
' . departments and Iqcav education agencies l^ve developed ' strate- 

gies for, implementing career education programs.- In the early 
years, Swanson .( 1971)" was addressing this . implementation"^ need r 

» ^ " ' . * ^ * ■ ■ ■ V. ■ - ' f ' » / ' 

Perhaps the most serious problem jjLn^^mplementation can- ^ • 
. ^- . be de^cribe4 ^s a ne^d for guidi|lz?t^ wh^ch permit local' 

/ education age'ncies^^,*^ smooth an^l orderly entrance ^ 

/ into career dievelopm^ent programs and^a way of anticipa:- - 
ting the costs of doing so. Such gifideTtnes might pro-- 
^\ pose organizational and structural alternatives available 

to schools, p^sgrinel requirements, for installing and', 
implement^ing plrogrami^, space - and, equipment requirements^ 
^or various program^ components and the degree to, which 
" career educatibn may, dey^l(3f|), "interchangeable parts for 

use within or among local/ education agencies. The need 
is for a g^'ide to sequencing' €he prlDceks of installing ' 
'\ ■ a career educa'tioni^ program. » r ' ^ 

" ^ The* desife* to Jsfuild flexibility into career education imple- 
^ mentation approaches arnd at the same tim^ remain accountable for 
the^ach'ievement of specif led o'bjectives btecame evident . Imple- 
k' ^ 'mentatidn strategy form^la^tion required judgments on the part of 
career, education cooi^'dinai;ofs and local project directors.'^ 
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irces of advice f6r thes3. individiials have efnerg^d. 
^Models^fdx implementing career7education mateAa:lg have been spe- . 
•cif ied *by Gros§ ajkd Kaplan; 1 1974 )'; . Shook and Morgan '(19727; Davis , 
Dwighfer Bofgen (l[974) ; the Maryl^ad S,tate Board of 'Ediacatidn ' 
^197^)-; McClure (jL975) ; Hoyt, Evans and Macken C1972) ; and K^eller 
(197'2) among: otheTrs; , The developers of the, "implementation, hand- ^ 
book.^or this project reviewed these ma^^erials in an effort to 
determine additional implementation approaches which would be * 
unique* and effective xjn t^e installation of career education pro- 
ductV. • - » . 



B'. The Need (for Programmatic Research Findings 

Mos'^s^f ' the information on career education implementation 
piractices was based on common sense and the opinions of dissemina- 
tion specialists- Implementation strategies based ori empiricaJ. 
datcf were practically non-existent. Some general -models of\ imple- 
mentation procedures 'do exist Zaltman and Other-s (1973) describe 
a seven stage process /of innovation acceptance- ThJ.s description 
includes the work of Koger and Shoemaker (l97l) and other theorists 
of change processes. Wpck by Kirkpatrick - (.1972) recognizes tJie 
need for ciefinitive ihfofma.tion and guidelines on strateg\(\ fornTu- 
•l-ation. He 'suggests that most judgments'^ which match tar'get audi- 
ence information with products being delivered currently are J^ei-ng 
made on, the basis of intuition ; ^^his' i^ occurring despite the 
investment being made in national R&D institutions such as'the' 
research centers and regional laboratories- - ' * 

I Rosenau, Hutchins and Hemphill (1971) in a concept paper for 
NIE state tjiat strategies designed to insu^re awarenes's .of products, 
and motivation fior product use may be the most difficult to engi- 
neer*. However, there is very little empirical evidence as to - ^ 

. what works best. This lack of i-nf brmation on how- to approactf pro-^' 
spectiye users- of R&D product information may have been .partially 
responsible for some of the role conflicts and difficulties ex- 
perienc,ed by education extension agents in diagnosing client prob- 
lems as reported by Sieberv Louis and Metzgisr (1972) . Undoubtedly,: 
some of Ehese problems could be reduced with the development of a , 
liandbook for diffusion agent's. I'he systematic c|evelopment of 

' product installation plans for particular' career education pro- 
ducts should facilitate/ product utilization. Crawford/ et al. 
,(1972) , have pointed ou't that R&D products which have been pre- 
determined diffusion strategy tend to be, ^accepted more quickly 
and with a wider distribution than those» which have a post hoc 
strategy- ' 1. ^ ; ' ' 

It was 'the pu^^p^e of this project, therefore, to develop a 
handbook . wh\ich , would project -directors in the formulation , of 

effective care^ education implementat;ion strategi'^s. 
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' The -Specifications for the Hanc^book ^ - ^, <; ' " « 

• The development of th'is career Education project directOK'^ 

handbopk was contingent upon* two important as'sLptfd^s: tlf the 

'aS^o? V^''^ reliable career education mat'eriai's/ " 

and (2) the critical nature of the project director's role/ These 
assumptions were critical to tte statement of implementation pro- 

o^fftrnf'''^''^'" '^^^ h^ndbook>nd the identl-f icatJorof a^ag?^ 
■ ?n^if user, audience.. Both of. these' assumptions were discussed 
.in detail during, the initial steering committee meeting'; See the 
notes froi^the^April 25-26, 1974 jneeting" of the steeri4 co^ittle 
in Appendix C for more informatioh. - ^ ^ conuniT;i:e$ 

. - ' ■ ■ • t ' ' " ■ ' ' ■ . • 

4,nitial set of specifications for the handbook Were as - 




:a. The CEPIH will contain tested and reliable infoirmation. 

.-(1) The CEPIH' will be based-qn research /rom the 

Diffusion progtam, other literature and discussion 
outside of the program, and inplit from- represents- ' 
tives of ^he user i population. * ' 

" . . . . ■ . . " ' ■- ,~ - ■ ^ 

(2) The GEPIH will be; used in actual and simulated ' 
career- education installatioh situations and ' " 
• ' assessed to determine .the extent to which indi- *^ 

viduals use the guidelines presented and form * 
strategies wlrich they^nd others 'perceive as 
- "^effective. ^ ' ■ 

b. The CEPIH will -be teievant for career edtiqation instal- 
lation acta^^iijiies . \^ • ^ 

(1) The CEPIH will be designed by (in part) and for • 
project directors or persons responsible' for - 
installing career education innovation. \, 

(2) The C^IPIH will contain -a •discussion and illustr/- 
tive examples on' how to develop. and initiate an 
installation strat;egy. This strateay will be * ' 
based on an a'ssessment ;of the (a) characteris- 
tics of the career education product itself - v 

° - ' (e..g. , content, cost, size, or resistance pp- ' 
tential) , (b) characteristics of the clients 
involved in, the acceptance and use o^ the pro-.' 
- , duct, and (c) a matching, and seqiiencing of' 

selected techhiques based,, on the above- informa"- 
tiCn. ' , • ' ^ 



(3) The CEPIH will also contain a discussion and/ 

a series of. steps on how to develop and implement 
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. ' a mechanism for proviaing feedback, from the ^client 
• ^ . or user ' s ' point «Df view, on the progress of the 
* *. \ ins-trdllation of §i given dareer education product. 

c. .The" CBP-IH will be easy to use . , 

, ^ ' *(1) The CE>PIH willi^be self-instructional . 

(2) The format of the 'CEP16 will be in .the fctrm of 
a handbook or reference to show project direc- 
tors how to formulate an installation strategy 
* for career education products. 

' ♦" ^ * ' 

' (3). The CfiPIILAvill contain a series of steps and 

* illustrative .examples to gilide/ the consumer 

through/the process. of developing. strategies' 
for implementing career education products . ^ 

(4) The CEPIH will have readable content in that 
' ' it will be free frbm jargon unique t6 any 

specific group of researchers or practitioriers . 

d. The CEPIH will be inexpensive . ; ^ 

(1) The CEPrH will be completely in printed form. 

"'^<2) TheVCEPIH wii:]. not require any resources for 

"^ts use oEhef than the intellect and ingenuity 
of the user. 

e. The CEPIH will improve an advocate's ability to/revise 

prodt3trt installation strategies" ^ ^ V" 

'. ~ 

(1) Pracixkioners will have participated in^the. 
(development of the CEPIH, and. assessments will 
have been recorded concerning the abiiity of 
the handbook to improve their formulation of 
diffusion strategies. 

(2) A laboratory simulation will have been "conducted 
=, :tcr experimentally test the effectiveness of the 

handbook with career education products. 

\, A handbook with these . characteristics should^ allow the 
formulation of an implementation strategy which recognizes the 
potential barriers1|to acceptance of^ the career education material-s 
establishes short range goals to -be achieved in the implementation 
process', and identifies ^relevant tactics for the achievement of 
these goals.; The .terminology used both to describe the handbook 
and within the handbook changed during the development process. 



This occurred as handbook users began to question the prescrip- 
tive, mechanical' nature of some of the processes. 

Discussions with field qite personnel by phone and in group 
meetings revealed negative feelings concerning t^e' use of the 
terms "product" and "installation." A softer more . judgmental 
decision process was deemed to be desirable. Therefore, the term 
"implementation" was substituted for ^other language in the title 
of the handbook and throughout its contents. The self -instruc- 
tional nature of the book soon came into question. Career educa- 
tion project directors at 'the summative evaluation meetings 
recommended the use of . in-servic§ sessions with individuals who 
are asked to use this book. / 

One of the major questions associated with the specjUEication 
of the handbook focused on the prime user population; ESly in 
the development of the project two user groups were clearly iden- 
tified:- (1) state coordinators of career education , and (2) 
local project, directors. Usually the local directors were located 
in local school districts, however, variations in size of districts 
resulted in very different demands being placed on. 4^hose directors 
in large districts compared to the smaller school districts- The 
question of whether or - nlDt to develop two versions .of the handbook, 
one for state departments and .one for local education agencie||, , 
was actively pursued during tTile development of prototype 1. In- 
terviews with state department personnejJ/- field site representa- 
tives, and consultants yielded information which suggested the . 
need for only one version of the handbook. This, version was to 
focus oh change processes in such a manner as to accomodate the 
different demands placed on project directors located^ in various 
types of LEAs . 



CHAPTER ,11 



THE DEVELOPMENT OF THE HANDBOOK 



A. Formative Engineering 

Thje formative engineering phase of this development process 
emphasized a sequence of tasks which led to the construction of 
the final protptype^o^ the handbook. Figure II-l illustrates the 
majfor c^e^fel^f5ment activities associated, with this project. The 
formative engiS^eering. phase exte^rided to' the development of pro- 
totype II. This phase allowed for the try-out and subsequent 
rejection of checklists a« a me^ns of formulating an implementa- 
tion strategy. It resulted in ^ section of the handbook being , 
used for workbook exercises. j * 

1. Development ;of Prototype I 

An important decision in the development of the implementa- 
tion of this handbook was the utilization of reliable findings 
from empirical research. Two research reports developed as pro- 
ducts from this contract were analyzed in depth for information 
leading to implementation processes and ' change tactics . These 
reports were: 

The Adoption of Systems Innovations in Educational 
Organizations: A Case' Study of Operation Guidance 
(R&D. No. 100) 

Perceived Effectivenes*s of Innovation Diffusion 
Tactics (R&D No. 9$) 

The findings from these reports as well as information from prior, 
research and development activities in the Diffusion of Innova- 
tions Program at the Center for Vocational Education was trans- 
lated into the structure and content of this handbook. Decision 
event 2,' discussed in Chapter III of this report, related to the 
translation of these research findings into the handbook. 

An attempt was made to glean implementation data specifically 
on career educai:i6h. One prior Center repqrt (R&D No. 96) and 
'case studies written by the six LEAs associated with the develop- 
ment and use of model I Vere analyzed carefully for information. 
Letters were written to pro ject directors at other sites* develop- 
ing career education models, two, three and four. Relevant 
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information 'was' exchan*ged. However, in;most cases, ^'nformation 
on* installation activities was not. V^t available. 

The publications cited in the r bibliography were revie^^d 
for details rfelatinq to implementation procedures- The general ^ 
models of implement^ion were valuable as guidelines in formula- 
ting ^general steps in the formulation 'of an implementation stra- 
tegy. The manuscripts which were specific to career education 
suggested unique areas of consideration for implementation stra- 
tegies. For example, the need for a definitive description .Of 
cf^reer education as aVconcjept became obvious . However, it was 
not. until the revision of prototype I that the developers inserted 
specific guidelines. This was due in part. to words of caution 
given by the .steering committee.' ^ . 

It became ^eces^ary to define and delimit the content of 
this implementation handbop'k.. Following communications with local 
,project directors and the steein>ji^ committee, the developers choose 
to write a handbook whlc^i would give ^specif ic suggestions leading 
to an 'implementation 'si;rategy. AnVattempt was made'^to omit mate- 
rials' which (1) would 'result in act^ivities generally considered 
to be outside the scope of project director '^duties^and (2), would 
be prescribed as part of a st^te education agenc^N^^lan for career 
education activities. An e>:ample, of the former .^bmission woUld 
be the inclusion of guidelines for selecting pilot sites ^n other,, 
school districts; an example of the latter exclusion would be ^ , 
guidelines establishing procedures for relating to the^'^bcal pro- 
gram to state agencies' plaVis. o " - 

/ The c^i'ticaJ pro ject^ director role requires many . judgmen|:al"' - 
decisj^ons../ . It --was, decided by the, developers to concentrat^yori 
the process of implementing career education material/, The^ 
handbook was to be viewed as a resource. book available far con- 
sultation as. the pro ject directors recognized problems and bar- 
riers to the. implementation process. This direction was, consis- 
tent with the advice received ^rom the steering coni^ittee meetings 
it was reinforced periodically as steering committee members had* 
the opportunity to review copies of protptype I and. express their 
advice . ^ / " 

Prdtotype I was distributed to field sites ip September of 
1974. J-t contained almost 100 single spaced pages, printed on 
both ^des. The .loose-leaf three-ring binder allowed for addi- 
tional comments or records of actions taken. .The handbook con- 
tained an introduction and 3ections on^planning , implementation, 
and assessing the impact of installation tactics. 

The prototype I handbook was supplemented with observations 
from the field via telephone and personal interview at the time' 
of decision event 2 (January 1975). However, the prototype' 
manuscript was very much in the developmental state. The format 





coiiid. change, and content was,/belng added, subtracted and sub- 
stjLtuted for existing sections of the document The decision. * 
event oh the translation of re.searc^h. finding^ posed several prob- 
lems: the research findings reviewed' in the programmatic 
reports represented in many cases obst^rved practice/ nqt neces- 
sarily ^durable solutions to persistertt^ problems. Likewise, ^ some 
of th^ (research findings were conflicting', in nature, thu^ posing 
problems for their translation into the handbook; the styles and 
fo:t^ats differed f rqm ^report to report ; although informaticfn at ' 
tilfnes tended to overlap from- report to report. The intent of trie 
eview was to identify key indicators of reliable knowledge which 
^ended^,.tO/.si3bsume much of the detailed res^rch findings. 

A& a result ofvdecision event 2 several changes in. the manu- 
script were- recommended: (1) illustrations were recommended for 
the procedural guidelines in the'book afs well as the :|.nstallation 
tactics which were identified in prototype I, (2) the^.developiers 
were- advised to be selective in their .use of reseafrch f indings^ - 
for translation into the book, |3) a glossary should be plafced , - 
in the front, of the handbjzJok, and (4) more items should-be con- 
idered as additional tadtics as the developers identify addi-., 
tional research findingsJ^ These recommendations as well as 
recommendatioTis from the field- sites^ and steering committee were, 
considered in the development of the prototype XI. 

After the developers, had incorporated many of the suggestion 
of the steering committe^; field site representatives^, and the 
decision event reviewer/ the initial draft of prototype/ II was 
given to a revision committee. This revision consisted of four 
individuals who reviewed the draft in advance of a review ^ssion 
conducted at Columbus, Ohdo. See , Appendix 'G for a ^mmary of the 
revision Committee mj^eting. ^ The output from this meetin*^ was 
incorporated into tjpte f JJial draft of prototype II which was dis- ' 
tributed to the field sj3:es in September 1974. / 

2. ^ \Selection and Use of f>±eld Sites 

Steering committee mel^bers were selected for the^t knowledge 
df career education activi|ties as well as other specialized roles 
related to the develof)ment) of ihstructiojaal ptoducts. Thus, it 
was logical to turn to them for advice on 'th^ selection of pilot 
states to -^ajj^^-ittTtbe of the haridlD'ook. Several states 

were nomiflated. "rneywere selected oil the basis of geograpnic 
location, the strength of their prograip it\ career education, and 
willingness to become a part of this dey^elopment programl r State 
coordinators of career education in Florida, Ohio and Texas were 
contacted by letter and telephone. Each state was asked to ^. 
identify, up to nine pilot schools which woyld be interested in 
usiug the handbook during a six months triaL period. Thes^ 
suggested guidelines • for nominating local education agenqie^ are 
contained in Appendix C. 



' Each local school district field site was contacted t1ie< 3^ 
developer and a meet ing scheduled to discu'ss the ^ntentiohs\ of 
the project; These meetings were held in each of the state.-s Uij 
^rly September with the loqil project dixectoj/ of c edyca- • 

tion. attending. No mohey was exch^^ed^betWe^n> any educ 
"agencies. The career education -^projec^dire^ the- handi- 

bbok for six -months, responded to tSleE^l^neViTiitl^rViews per 1^ 
cally, and returned^a mail questionnaireOeya^Q^ating prototype I 
at lihe concLusion of the try-out period.:^- in addition, these *. 
field site career educa.tion\directors helped ptoject staff ^ select ' 
simulation experiences which were used in the sumrnative evalu*^- 
tion. <9 . . n . 



The field sites selected repres^^nted "^^range of conditions- 
Thev^ included urban school v^istricts^ ^s' well asj- smal\l rural^ dis- 
tricts. , Ethnic groiios were represwit^d\ in many 'of the sites. 
Each site had previously developecjl career education products for 
use ,ijn implementation. These prod^icts ranged frofn vocation'al 
guidance placement procedures to the use of Specific curriculum-, 
guides for infusing care'er education. / • 

.Careert education '^roj ect directors in f ield sites were eager- 
■■'^ to cooperate. A'total of eighlrsite visits were conducted by 
proje.ct staff and thirteen pfedne interviews were held. One of 
these,', interviews was'^a "conference call to the state education / 
agencies and other sites. Specific attention was focused on the' 
question of whether* or not to develop a\, special version of the, 
' handbook for career education coordinator s in state departments. 
All bu.t one of" the telephone respondents .reported using the hand- 
book' d,ur ing the six-month, per iod . However, most of the use seeme^d 
, to take: place at , the b.eginning of the project. As time progressed 
' pro ject director sNbecame distracted with other priorities. ^The 
least experienced dijirec^Tors tended to use the handbook more than 
the experienced ones!^ . 

The planning section seemed to be of greatest v^lue to the 
locaT directors. However, they also liked .the strategies section 
dnd tended to agrefe with the need to record their pl^ns . Very ^ 
- few of these project directors actually used the checklist, how- 
ever. • There appeared to be a need to, change the format of the 
planning section. 



The directors were asked to identify -anV other materials 
similar to the handbook which they^found useful. Some were 
able, to, gecall some titles or the name of an. author. But in . 
general;, very fe,W knew knpwn resource books werp identified. 'The 
telephoi'rie interviews were only moderAtely helpful in obtaining*' 
suggestiions for changes^ in the handbook. This was true because 
the developers were. unaJDle to sustain a continuing dialogue ^.with 
the fcield site project directors over time. Also, the developers 
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had a lacJOrof knowledge about ^the' unique implementation problems 
occurring on^srte&/ Whenever possible^ project directors were 
aler^d Vin advai^ce' to the telepbohe calls. This seemed tto help - 
th^m (Otganize their thoughts f'or.'tl^e questions. 

■ In general, the responses on the mailed cmestionnaires were 
more helpful in revisi/ig the content of J:he^ handbook.. than '^the I 
telephone interviews. Eighteen' of the^twenty LEAVfield sites^ " 
^^espohded A'The results of this formatiVe evaluatior^ are contained 
in AppendiTc^D. They tend ;to indicate that a Vid^ -.variety of 4pe3;- 
sons are .likely to be able tc benefit' fronj receiving the ha^idboolc. 
However , administrators , local supervisors ) and cur |dcu rum/deve- 
lopers are most likely to pbtain maximum berj^ fit. The inmrmation 
in all sections of the handbook- wa^ considered to beV'usef lil by 
alimpst '^ll respondents'. It was easy to find and contained yio 
serrous social ^Dias.es. , The ' terminology was easily understood by ' 
most people and tHe org^iization of the material was meaningful. 
They said the handboc5^'^'''eHt5Quraged^ a planning of . installation, 

strategies although this section needed improvement. 

" ^The respondents to this formative eyMttation were mostly 
.between. 3 0 and 4 5 yjears^ of age with- a m,a4tfeVs degree : or above ^ % 
assigned pritnarily to administrative ^br(su(pervisory duties. ^ 
Most ofUthe^pro ject directors "had spent six yefars of less in 
assigninents ^simila?: to th^r present ^ne. These individuals;, were 
able to- brirtg their years of experience to bear on problems 
associated with developing t^e impl^entation handbook. _ Their 
suggestions were Written into the revised prototype .and reviewed , 
by the revision'^ committee.' * ^ / ' ^ 

B. SijiTunative Engineering _ ' , 

This phase of the - product? development compares the revised 
handbook ^ prototype with materials ^already on the market. The 
book by Ronald Havelock entitled^ "The^^hange Agent's Guide to 
Innovation ifi Education" was selected l^errause it has been widely 
accepted throughout the nation. Dr. Havelock is a noted authority 
in this area. The content 'of the book parallels the process steps 
developed in: the career education, implementation handbook. How- 
ever the Havelock book is not specific to career education. 

. The individuals attending ,the summative evaluation-meetings 

were randomly selected from throughout the ^nation. Thq details 

^of this selection are' contained in Chapter III. Most of the 
indM;'idiaals were carder edupatibn project directors in local 

-education agencies. However, ^ome were operating on special 
projects at the state and district levels.. 

An audio-visual presentation was ;shared with this group of 
local project director^ for the purpose of obtaining their 
^suggestions for impro<^ement . ' This presentation was designed by 



the developers to aid in the in-service orientation of * persons » 

using the implementation har>dbook. However, care was ' taken during 

the summative evaluation th^t/this audio-visjaal presentation not ' 

influence the results of tKe -experiment. 



/ Suggestions for changes *in the revised prototype were^solic- 
iteci>^^^oiji participants in the 3ummative evaluation. Likewise 
suggestions for revision 'in prototype 'II were solicted from mem- 
bers of the" steer ing^^ommittee . . Some of the summative evaluation 
participants' comments follow: ^. 

"I used 'the handbook but not* to a ;glreat extent because 
I was drawing mo r^ upon p^me of my own .experiences . " 

. ^ . "There- was no way I could just use th^^ handbook without ' ' 
putting me into it and how I best f^nc!:tio'n; " 
■• , . ■ ^ _ ' . ' , ■ - ■ , 

♦ /'Not * having as. much experience ;!^"S fnost af you prgbably ^ 
have, I;, on the other hand, went directly to the^ book." 

^ V-f - , ' ... ■ . - 

"The whole p^rpcess did make me tend to zero in on^what 
is my problem. . So.'of-teh iiL is so easy .to. -think aBofut 
' , .possibl^e solution:^ before /you really identify- that- 
problem." , ' ^ ^ ' . 

"The handbook really heeds some in-service." 

••- , - • ■ . - ■ ^ ■ I. 

I don't quite see the correlation between the ^ \ 

information in front of the book and- the. tacl^cs." ^ 

^*It (handbook) got too deep, too technical*, too 
quickly . " , 

"J'm fairly pleased with the material 'iji here.- It . 
turns out that a lot of the things I d6-,because of? 
experience are wiritten out." * ' . ^ ' 




"I'm very comfortable wi|:h ^the 'material in it." 

"If you Ho have something like this, you almost 
• need some type of session like we ju5t went through." 

"I used it as a reference^" ^ 

The Havelock book was a good, ch9ice as a comparison trelatmerit 
because it contains a, section on installation taptics and sJtrate- 
gies. It has been in use throughout the nation and has b^n ' ^ 
revised. Like^wise , the book ii^ available commercially. ^ Books ^ 
relating specif ically to , career education were v^ot selected bje- 
cause all that the authors identified were specific to a particular 
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concept of career education being promoted in a particular st\te. 
Alsp, these statd^ handbooks tended to focus on the total spectrum 
of career "education advocacy and did not pinpoint tl}e formulations 
.of an implementation strategy. It is interesting to note th^t 
the definition of pareer education was omitted from prototype I 
based on thjs advice of the steering committee . - Later this recom- \ 
mendation was reversied by results from the field use of the hand- 
book and questions . received from summative evaluation participan^Sk 
These participants had^^pecific suggestions such' as reversing tlgjj^^ 
order of the first two steps in the handbook. Also, some! of them 
were opposed to the use of "coer tactics. But ^ in general, 

the handbook was well received and many participants. were reluc- 
tant to give it up at the end ^of a session. They were promised 
complimentary •copies at ^th« conclusion of the project.^ 




CHAPTER 



Evaluation 




The evaluation employed 4n this project addressed four major 
question^. These questions included t'he following:' (1), Are .the 
two research rej^orts .^ptoduqed ^in Work Unit A ready foa; . release? 
(2) Are the" research^ findings incorporated into the handbook 
effectively? (3) , Is the handbook' in a form useful to local 'career 
education directors? and M) Does the h^ndbopk^^ignif icantly 
affects local career education directors' performance over and ^ 
above that which would , occur by use of a similar product?- This 
section disp;asses the evaluation related to each of these major 
decision points. ' ; • . 

. Decision Point 1 

^...^w decision point addresses the .question : Are the two 
jresearch reports produced in Work Unit A for release? These two 
reports included: V • . 

Hull, William L. and Kester, Ralph jP". Perceived - 
Effectiveness of Innovati^on Diffusion Tactics ,: The 
Center for Vocational Education, The Ohio- State 
Univei;sit^, Columbus, ^hio, 197 5. ; , / 

Kester, Ralph J, and Hq];;^ Jr .\ The \ 

Adoption of Systems Innovations in Educa-tional , 
Organizations: A-^ Case Study of Operation, . 

' Guidance , The Center for Vpcational. Educatian , . 

• The Ohio - State University , Coluihbus , Ohio, 1975 . 

Thisvdecisio'h' point was subsequ^ltly divided ^ntp two. cJecision 
tevents (one for each. publication) . The' first decision event was 
held on November 8, 1974. Two reviewers were— aisked to review 
the publication. The Adoption oig* Systems Innovations in Educa - 
tional Organizations:' A Case Study of Operation Guidance , and ^ 
make recommendations related tOj,.publication. The overall 
recommendation for this decision event was to make' some modifi- 
cation and publish the .study, i 

The second decision, event was. held on December 12, 1974 . 
Two ' revdewers Were employed to examine the draft publication', 
Perceivedv Effectiveness of Innovation Diffusion Tactibs, and 
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itiakeva determination of its readiness for publication. The/basic 
o\/tcome of this ray^iew was that the publication be published with 
minor modification. - - ' - * 

, . ^ ■, . 

' ^ . ^^--^ Decision 'Point 2 • ' . V 

v The second decision point was concerned ylth determining the 
extent to which the research findingsi were incorporated ^nto the 
CEPIH. A reviewer wasfcjselected . and this deci^ipn point was held 
on JaiinJiary 28-29, 1975..' The overall 'purpose of this'' d^ecision 
point was to assure that the handbook wari. based -on vempirical^ evi-- 
d^nce and that the general^iatfens made in the handbook were"^ • 
based upon soiand judgment . This ' decision point 'designed to 

provide'infprmation to the program personnel oh thd adequacy af 
these incorporation jef forts. TKe' ba si <r^ outcome of this decision / - 
pofnt was that some modifications be made in the incorporation 'j; 
of , the findings ar>d that some other additional findings be i. ^'>. 
incorporated. . " ■ • 

Decision Point 3 

This decision point was designed jto collect information on 
the usefulness. of the CEPIH to loteal career education project 
directors. This evaluation was". coViducted through a -field trial 
of the inilfial draft of CEPIH in the thr;ee states of Floislda, 
^TOhio and Texas. Cppies of the- initial draft wer^ provided to 
^e^ch of the three state career education directors and to a total, 
of 20 local career education directors in the .three statfes. A 
list of these individuals is « provided in Appendix A. A map,- 
showing the -geographical locations of these indi'Viduals is 
.presented in Figure. III-l; Each of these individuals were asked 
to use the CEPJH from September 1974 to i'ebruary 1975 as they * - 
dealt with career edii^ation installation problems. In February 
-1975/ e.ach of the career education directors were asked for their 
comments and suggestions for improving the handbook. The areas' . . 
specifically addressed by the final trial were: (1) usefulness 
of :^he CEPfH, (2)^ ease of finding information, (3) seri9us biases ' 
(e.g., cultural, sexual, economic, etc.), (4) "ease of understand- ^ 
ing^ (Sy meaningfulness of material, and (6) the degree to which 
CEPIH encouraged actiye planning of installation strategies.. Ad 
a result x>f this field trial, CEPIH was substantially revised to 
;be .in a form more acceptable to local -career education directors • • 

-* ^ ' - - *' I ■ *' ' . . '■ 

Decision Point, 4 

This decision point was designed to obtajln information rela- 
tive to the final summative evaluation of the CEPIH. Data to 
answer the question, "Does the handbook 'significantly affect ' \ 
dififuseEs of career education product performances over and^ above 
that which would occur by a similar product?'^ ^was collected by ' 
the Center. The plan for obtaining this }.nformation was the use 
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of 'a 'pre-post-tesf control group design suggested by Campbell 
and Stanley (1963). The basic overall design is represented as 
follows : ^ 
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Each state director of career education was asked to icfen- ' 
tify local directors of career education for state funded career 
educdtion projects in their state. It should be noted/ however, 
that states involved in the formative evaluation efforts (i.e., 
Florida, Texas , -^nd Ohio) were not included because of their 
previous involvement with CEEIH. Additionally , the states of 
Hawaii and Alaska were not included due to the fiigh travel cost^ 
of personnel from the^^ two states. From the •lists of local 
career education pro ject directors submitted , a total of 56. dir- 
ectors were randomly selected to participate in this summative 
evaluation. 

Local; career education ^directors with three years ot less of 
experience were assigned to level A^ ; local directors with more 
than three years of experience were assigned to ^^level A^f. One 
half of the local directors with three years or less ex/erience 
directing career education pi^jects were then randomlyyassigned 
to treatmenti (GfiPIH) and the remainder were assigned to treat- 
ment2 (The Change Agent's Guide to I^inovation in Education). 
The. local directors with more than thr^e years experience were' 
randomly assigned in a similar spanner. ' 

The list of Ibcal^career education directors who 'participated 
in the summative evaluation is given in Appendix A. A map showing 
the geographic location of these local directors is included as " 
Figure III-2. -Attrition due to last minute chfanges in plans, • 
availability of directors, etc. resulted in a total of 46 local 
project directors in the total. \ . 




G? 



Rir 



29 



:l__-_J : 

Figure III-2,. Geographic Location of Local Career Education Directors 
^■ ^ , Who Particij^ted in the Final Suranative Evaluation 
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Demographic varijibles associated with the local career edu- 
cation project directors is presented in Table III-l. The major-/ 
ity of these directors had a Bachelor or. Misters Degree (76il .I 
percent) , were employed in a locail education agency (76 . r percent) 
for three or less years (60 . 9 percent) , had served as a pro ject 
director for two^or less years^ (69.6 percent) ^and had one year 
or no expferiencd* directing, other project-type activities '(-54.4 
percent) / ^ ^ , . \ , , ' - 

The: multiple-choice career education product installation 
cognitive test was developed for use as a pre- anii post-test . 
The original test consisted of 80 items and was pilot-tested with 
a' group of graduate students at The Ohio State University . As a 
result of this pilot test the .cognitive test was reduce^ to 50 
items. ... ^ , ' 

|fc^Thi^ internal consistency reliabilitij^ for the cognitive test. ' 
was^ estimated utilizing %he pretest data obtained from both 
"treatlmenti" and "tfeatment2" groups. The internal consistency 
analysis utij^lzed the .Kuder-Richardson 20 and Kuder-Richardson 
21 formulae :as illustrated by Guilford (1965, pp. 458-462). This 
analysis resulted in a Kuder-Richardson 20 reliability of .56 and 

Kuder-Richardson 21 reliability of .44 . A similar procedure was 
Utilized with a group of 41 local qareer education personnel from ' 
Western Michigan. This ^analysis resulted in a Kudejr-Ridhardson 
2i)f reliability of .71 and a Kuder-Richardson 21 reliability of 
.63. ^ - 

The saitie test was used for th^ pretest and post-test . How.- 
ever; two different versions were used. The difference in the . 
versions was the sequence of iteiris. One half of the group of 
participants received the first sequence for their' pretest . The 
other half of participants received the other sequence. The ver- 
sion of the test was reversed during the post-test phase. A copy 
of the cog^nitive test is provided vih Appendix B. ^ 

The test-i^etest reliability was computed from a comparison 
of the pre- and post-test scores for the " treatment 2 " g^roup only. 
This analysis utilized a Pearson product-moment correlation of 
the two tests. There was some confound^ing of the test-retest I 
reliability by the informa'^tion .being given to the "treatmento" 
grotip due to the use of "The Change Agent's Guide to Innovation 
in Education." However., this shojuld make for a more conservative 
estimate of the test-retest reliability. This analysis resulted 
in a testrrete^t reliability of .79. 



TABLE III-1-. 



DEMOGRAPHIC CHARACTERISTICS OF LOCAL CAREER EDUCATION DIRECTORS 



Demographic Characteristics 



Number -V 
of 

Re'spondents 



T ' Percent 
' of 
- Respondent^ 



Educational Level 
^ Bachelor 
Masters' 
Specialist 
Doctoral 
* Total 



Present Employer 

Local Educatiqn Agency 
Regional Education Agency 
State Education Agency 
College or'' University 
• Total 



5 
30 

2 
_9 
46 

35 
8 



Years Experience With Career Education, 
1 . ^ 

■ 2 . .. . . ' " ^ 

3 , , ■ 

A . . • 

. 5 . 

.10 ' . 

2A ■ . 

Total ' - 



.1 
>10i 

17 

10 
6 
1 

_J. 

46 



Years Experience As Career Education 
Project Director 

1 ■ 13- 

2 ' 19 

3 ' . 10 
^- , It 2a 

5 1 
10 ' _± 

Total 46 

Years Experience Directing Other Projects 

0 '20 

' 1 5 

, 2 .' 6 

3 . 7 

4 , 3 
■ 5 ' 2 

6 • 1 

7 ' ■ 1 
25 1 , 



10.9 
65 . 2 
4.3. 
19.6 
100.0 



76.1 
17.4 
2.2 
4.3 
100.0 



2.2 
21.7 
37.0 
21.7 
13.0 
2.2 
2.2. 



100.0 



28.3 
41.3 
21.7 
4.3 
2.2 
2.2 
100.0 



43.5 
10.9 
13.0 
15.2 
6.5 
4.3 
2.2 
2.2 
2.2 
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A Cognitive Test of Career Education Product InstallationyvKnovljedge 

In the sxuninative evaluation the researchers were interested in 
. evaluating the effects of experience (factor A) and two \handbooks 
(factpr B) on care^r^^cJucatdon product installation knowledge as 
measured by a cognitive test. The summafive evaluation consisted 
of a simulated use of the handbooks for/a three day period. A . ■ . 
copy of the workshop agenda is presented in Appendix B. The local 
directors were administered^ a test on their cognitiV^e knowledge of 
career education product installation on both a pre- and post-test 
situation (factor C) . The criterion measure was the number of 
correct -responses on the pre- and post-tests.. ; ^ 

The mean cognitive pre- and post-test scores for the suiranative 
evaluation of/the CEPJH is presented in.Table III-2. The data 
. reveals that the individuals who received the CEPIH had a mean 
30.76 on the pretest. In contrast^ those individuals who utilized 
The Change Agent's Guide, to Innovation in Education had a- mean pr e- 
test score of 3I..38. \A statistical analysis utilizing- a t-test 
as suggested by Winer (1962, p. 242 and 344) revealed that there, • 
was no significant differerxce between the mean pretest scores of 
these two groups (t-value. = .46) . This finding leads one to the 
conclusion that there >was no statistical difference in, the cogni-- — 
tive knowledge of career education product installation between 
the two groups prltor to the experiment. 





TABLE 








MEAN 


COGNITIVE PRE" AND POST^TEST SCORES- BY 


.TREATMENT 


GROUP 


Group 


Pretests 


Post-rtg^t 


> Gain 


Average ' 


: n ; 

Treatment 2 


30.76 


34.18 


-3.42 


31.86 /\ 


Treatnient2 


31.38 


32.34 


.96 


32.47 


Average - 


31.07 


.33.26 







. _ The statistical technique followed in analyzing_this data was 
a three factor e;«periment with repeated measures as v^syggested by 
Winer (1962, 0^^^337-344). The; analysis of variance is summarized 
in Table lljf^. The .05 level of significance is used, in this 
.analysis. In this analysis, the main effect for factor C (pre- 
posttest) is found to be statistically significant beyond the .01 
level. This ifidicates the mean pretest score is statistically ^ 
significant from th|& mean post-test score. Inspection of the data 
in Table III-2 revealed a mean pretest score of 31.07 compared to 
a mea^ post-test score of 3^.26. This finding indicated that the 
participants gained a significant amount of Knowledge on career 
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education product installatiorv from the time the experiment began 
until, it ended. The treatmen-ft X pre-post-test interaction was also- 
statisticaily significant (at the .02 level). This interaction 
indicates 'that the pre-post-tifest scores differed depending upon 
the treatment. A graph indicating this interaction effect is 
shown in Figure III-3. This graph indicates that the effect of 
the two handbooks on the , knowledge level of career education pro- 
duct installation differs by. handbook. - 

TABLE III-3 

. ^ SUMMARY "^F ANALYSIS OF VARIANCE FOR THREE FACTOR E}^ERIMENT • t 



Source ^ 


Sum of Squares " 


■ df ■ 


Mean 


SquAre v 


,' F Ratio • 


Experience' (A) 


29.87 


1 


29: 


;87 . 


.87 


Treatment (B) 


mk 8.12 


1 




,12 


.24- 


AXB _ ^ 


.25 ■ 


1 ■ 




.25 


.01' 


Subject (D) 


1443.58 


42 . 


- 34, 


.37 




Pre-post-test (C) 


106.41 ^ . 


1 


106, 


.41 


18.29** 


AXC 


.44 


1 




.44 ■ 


. 08 


BXC ^ ■ 


3,3.24 


1 


33 , 


.24 


5.27* 


AXBXC 


6.91 


• 1' 


6, 


.91 


1.19 . 


CXD 


244.30 

c 


42 


5; 


.82 





A^-^Signif leant beyond the .01 level 
* Significant beyond the .05 level 
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Furthermore, those individuals who received the CEPIH gained more 
than those who received The -Change Agent's Guide tolnnovation in" 
Gareer Education. 

Simulated Situations . " 

In addition to the p.re- and post-test of career education pro- 
duct installation knowledge, the local directors were also given 
three simulated situations to which they were asked to prepare a 
written response. Copies of these situations are included in 
Appendix B. < ^.^^ 




Th^se simulated problems were developed on the basis of prob- 
lems actually encountered by local career education directors 
{Kahler% personal communication) . , A total of 30 situations were 
developed^ and submitted to 'the career education directors involved 
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in the field trial af CEPIH. -These directors rated, ea of , these 
§>ituations as to their realism. Fjrom the 30 situations^ nine were 
^selected as being most realistic w -Finally, thf^^e' situ^ions were ' 
selected* to be included in the sujranatiVe ev'alua^tiofi . These cthree 
situations- represented different stages in the instaIlatioi1^;pro- 

cess (i.e., early , 'middle , and l^te); , ' . . • * vV. / 

• 'I ' ■ . ' • . ' ■ 

Each of the siinulated Career education implementation problem^ 
W^fe used by each Iq'car director to formulate an ^implementation 
strategy for each prbblem; ' Each local director was allowed one 
•hour in which to prepare Jh,is/her reactions. Following this, small 
group discussions were h^Id for one hour which allowed the loc^l 
directorjs to get feedbadk from' the other local career education 
directors on their solutions and how the handbooks were utilizecJ. 

\ Upon completion of the simulated f ield,,. trial , three judges were 
selected to evaluate the local directors ' responses . ' These ^ judges 
were knowledgeable -of .career Education , school administration, and/ 
or diffusion of innovations. Each judge was asked to rate the 
written response on a 5~point scale (1 = low, .5 - high) on the 
basis of: (1) ci^airity of communication, (2) logic of approach, 
and (3) chances of success. Additionally, each judge was asked 
to place no less, than 5 responses in any one cate^o^.- 

Th^ inter-rater reliability of the. judges, was a major concern : 
m the analysis of these data. This reliability , was established 
utilizing the Analysis of variance technique suggested by Wi>ner 
(1.962^ pp. 124-132). The formula fo;r determining this estimate 
of reliability is as follows: , * ' 



..r = 



MS 



between peopf e. ■ - ^"^^resadual 
^ ^ between people 

In utilizing thi^analysis it is assumed tfiat : • (1) the error 
of .measurement /associa^ted with the simulated problem is uncorre- 
lated with the true score, (2) the local directors were a random 
sample of all career education directors , (3) the simulated prob- 
lems were a random sample of problems .local career' education dir- 
^^tors face, and (4) the wi thin-person variance could be pooled 
t9 jprovide an estimate of the error of measurement, variance 
(i.e., that there were no person by situation^eff ects) . • * 

Chi-square analyses w^te then conducted to determine if the 
scores of , the local career education pro ject director s on the 
simulated problems differed by treatment. The formula utilized 
in cqmputiTig chi-square values was obtained from Snedecor (1967 , 
p. 30) afRd is as follows: 



. , TABLE III-4 

JUDGES' RATINGS OF RESPONSES TO SITUAqCION 1 BY SUBJECT 

.Subject's Score • 

Subject Judge 1 Judge 2 Judge 3 Total / 





1 


3 


1 


4 


O ' 

O 






2 ' 


3 


2 « 


7 






2 


2 


3 


7 






c 


3 


3 


11 




D \ 




4 


<5 
3 


• 11 




o 


3 


c 


3 


-'.II- 




/ 


3' 


1 ■ 1 ' ■ ■ ' 

\ ' 


.2 


* ■' * i: 
Q 








\ ^ 1 ' * 


^. J. 


3 . 






/i 

H . . 


V 1 

■ V- 




Q • 




J.U 


' ." 




Z 


0 




J-J- 


' . ' -c ■ 
J 


c 
' D 


■ » 

c 
D 


X J 




* 1 9 


o 
J 


J. * 


1'/ 
J 






. . t 




9 
Z 


A 


J- J- 




^ A 


* ■ 1 


1 


J. 






J--) 


J 




9 
Z 


Q 




f 6 


• J 1. 




'A 


1 A 




J./ 


Z • ' 


9 * 
Z 


Z 


di ■ '"A 




1ft 
J.O 


* 'X 

J 


J. 


J. • 


c 




TO 


c 


3 


c 
J 


. 13 ' 






3 


1 


3 


7 






r 3 , ': ; 


1 


2 . 








3 . , ' ■ 


3 . 


2 






23 . 


c '■ 

5 


5 


- . 2 


no " 

12 








1 


1 


6 








1 


2 


1 




20* 


2 


1 


1 


i \ 




2 / 




2 


/ I 


\^ 4 




2o 


* 2 r 


3 J 


^ 2 


1 




2y 




J / 


Z 


J *xl ? ^ 




jU 


. 2' .: • \ 


Zi / 


. : J- 








-3 


A ^ 


<- J- ' 


■ _ ■■ p- . 






1 

J- 


1 

J- 


9 








■ i, ): 


J- 


9 

z 








9 


9 


J 






•• 35 




1 


3 


7 




36 - 






5 






37 






9 


a 

v 




38 


1 , ^ 


3 


3 


7 




- 3,9 


3 


■ 1 


3 


7 


i 


40 


: 4 


2 


2 


: ■ ■ 8 




> 41 - 


2 


2 


.1 


■ ■ .-u 5' 




•), 42. . ■ 


5 


• 4 


5 


14 . 




■ 43 


4 


-2 . 


. 1 , 


7 




' 44 


1 ' 


1 " . 


2 


4 




45 


■■ 54 


' 1 


4 ' 






^ 46- 


4 


3 


■ .■ .2 


9 


Average 


Judge^ Score ' 


3 . 02 


2.30 


2.57 
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Situation 1 ; . . • . ^ 

• ■ '■ ' 

•The judges' ratiiig of the^ local career education director's 
written response by subject., is presented in .Table III-4. These 
data were thien analyzed utilizing the analysis of variance tech- 
nique '^suggested by Winer.' The summary table. of this analysis- of 
v4Tiairce is. provided in- Table III~5. ^ 

' • / TABLE III-5: , \ • V . ' 

SUMMARY ANALYSIS OF VARIANCE TABLE FOR SITUATION 1' ■ ^ 



< ■ 

Sources of Variation 


S'S. ■ 


df.. 


— — J 

MS 


Between people 


135.48 


45 


3.01 


Within people ' , 


llZ'i'e? 


' 138 . 


.82 


.Between judges ' 


12.-13 


2 ; 


6.07 


Residual 


. : > 100.54 


136 ' 


.74 


* Total \ ■ ^ ■ 


, 248.15 


183 





The- inter-rater reliability computed ^fromdat^ in Table IIi-5 
using the formula suggested by Winer resulted in "an estimated re- 
liability ;df:^ This estimate of reliability re'f^rs to t'he 
'kv^etaqe of tiif three ratings made.on each o^ the 46 local career 
■education V directors.' Another way of stating 'this would be thajt 
if .the experiment were to be perfprmed again with another^ample 
of four judges, l^ut with the same' local directors , the correlation 
between the mean rating^ obtained from the two sets of *data wpulcj. 
be approximately .75. ' ; . 

. The ratings of the three judges were then combined to prov^^e 
a more stable estimate of each subject score. Table III-6 pr.c^B&e 
a summary, of the total scores by treatment graup.. The chi-squ'^ie 
value' needed for signifapance' at the .05 level (one-tailed /de^t) 
wttTi : 11 degrees of ;fr 19 . 7 . An analysis of the data re- 

sulted in a chi-squalre value of 14.96. "^Thl§ value was less than, 
the' level ' needed for' significance, therefore, there/ was no statis- 
tical differehc^/between the scores on the simulated problems pf 
the local career education director In "treatmenti" and those; 
local career education directors in " treatment2 " . ' 

^ityation 2 : 

The judges": rating of the local career education director *s , 
written response by subject ;i$ >pf^^ in Table III-7. The 

inter-rater reliability for Situation 2 was conducted utilizing 
the same procedtjre a^ Situation 1. A Summary analysis o£ variance 
table for this dai^^s shown in Table III-8. The inter-rater 



TABLE 111-^6. 



NUMBER lOCAX CAREER EDUCATION DIRECTORS ^BY TOTAL 
V ; . ; SCOJt| AND T]^ATMENT FOR SITUATION 1 



Totk^l Score 



Niimber pf Local Career Education Directors 



V Tr^^Sment^' 



Treatment, 



Totiir 



3 






0 ' 


2 

- iU 








2 


•* , , ; „■ 

• .5 . 


■ ■ 5 • 




0 , ; 


V- ' 2,.:: .' 


2 


6 

f. 










7 , 




7 




'-'>' . ■ 'II./-:- ^' 


....■8 / 










9 




3 


2 


> ' : ■ ^ 5, . ■ ■ 


10 . 




0 


0 


0 - 


• 11 :'. 








- ' ' 5 . , 


12 
13 






■ 1 ■ ' 


■■■ \ 

1' 


14 




0 


2 _ 


2 . 


15 




0 , 


' " . 1 ^ 
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JUDGES ] RATINGS OF RESPONSES TO SITUATION 2- BY SUBJECT 



Score 



Subject 


, Judge. 1 


, - Judge 2 


'Judge 3 


Total 




"'^ ■ 
J 


1 
J. 


2 


6 


9 ! 




9 

•* 


/ 3 


10 


J 


"2/ 


A ' 


L 

•-r 




A 


A 


- 9 
z 


J 


Q 






c 
J 


A 


1 1 

X X 


• It 


: J 




J! 


Q 


/ 


J 


1 


9 


D 


0 ; 


J 


1 
J. 




? D 


Q ' 


/, 


9 
z 




' 1 1 


1 n 




J 


9 


q 


■LX 








1 S 


1 9 


J 


'X 


3 


Q 
y 




'I 
.J 


~J 


5 


11 


1 A ' 


A 

*-r 


1 

J. 




9 


1 S 


A 


, \ 




9 


1 




c , ' ■ 
^( 


5 


JL J 


17 


"\ 

J 


0 


3 


Q 


18 ' 




J. 


3 


7 


1 Q ^ 


• 5 


J 


5 


15 


9n 


9 
z 


1 

J. 


2 


5 


9 1 " . 


9 
Z 


1 

J. 


9 


J 


9 9 


J, 


1 

J. 


9 


D 


9 




-5 




1 


9/1 


9 
Z 




'1 
J. 




25 


3 


1 


Z 


6 


9 A 


J * 


9 

■ ^ . Z 


9 


7 


9 7 




- H 




in 


9 Q 
Zo 


z 


9 
Z 


J 


7 


9 Q 

zy 


1 
i 


J 


9 
Z 


D 


jV 


■ 1 
i 


9 
Z 


9 

z 


c 
3 


1 1 


1 


1 
i 


1 

1 


J 


J Z - 


1 

i 


1 
i 


1 


J 




1 


J- 


9 
Z 


A 


' 3 A 


0 


9 ' 

z 


A 


Q 

y 


\ J J 


1 
i 


1 


1 

J. 






' J 


A 


'\ 
J 


1 n 
xu 


37 


1 


1 ' . 


■ 1 


3 


38 


2 






4 


39 


-1 / 


3 . 




7 


■ 40 


1 


: 3 ^ 


5 


9 


41 


1 


1 


2 


4 


42 


5 


■ A . 


4 ^ 


13 


' 43 


A 


2 


2 


8 


44 


2 




3 ' 


9 


45 


2 


2 


2 


6 


46 


3 ■ 


1 


^ 3 


7 


Score 


2.76 


2.35 

29 


2.80 
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reliability was computed following the same procedure as Situation 
1. Thisscomputation resulted in kn estimated inter-rater reliabil- 



TABLE III-8 

SUMMARY ANALYSIS OF VARIANCE TABLE FOR SITUATION 2 



Source of Variation 

? 


SS 


df 


MS 


Between people- * ■ 


153.88 


. 45 . 


3.42 


Within people 


82.00 


138 


.'59 


.Between judges 


5.84 


2 


2.92 


Residual ' 


76.16 


136 


.56 


Total 


' 235.88 


183 





^The researchers again combined the judges' scores and conducted 
a chi-square analysis to determine if the treatment groups differed 
in their ability to solve these simulated problems. Table III-9 
provides -a summary of the local career education directors total 
judges sd^ore for Situation 2 by treatment group; 'The chi-square 
value needed for significance at the .05 level with 10 degrees of 
freedom was 18.3 (one-tailed test). A chi-square analysis of the 
data resulted in a value of 11.9 and Vas not a significant value. 
Therefore, the two treatment groups did not differ statistically 
in th'eir ability to respond to Situation 2. 



TABLE III-9 

NUMBER OF LOCAL CAREER EDUCATION DIRECTORS BY TOTAL SCORE AND 

TREATMENT FOR SITUATION 2 



Total Score Number of Local Caree r Education DlrectOiTs 

Treatment-j^ ^Treatment2 ' Total 

3 ^ ■ 2 ■ '2 . ' 4 

4 0- , , 3 3 

5 2 13 

6 . 5 3 8 

2 3 5 , 

8 . 1 0 . 1 ■ 

9 7.3 ^10 

10 2 2 4 

11 • 1 ,.. ■ ■ 2 3 
13 • ■ . 0- 2 2 

'15 0 ^ J . 



situation 3 : 



The local caireet education directors' scores ofi Situation 3 
by judge is presented in Table IIl-lO... The inter-rater ^reliability 
for this situation was conducted utilizing the same procedure as 
the two preceding situations, A summary analysis of variance for n 
Situation '3 is presented in Table III-ll, Again the intejr-rater 
reliability was conducted utilizing the same procedure as- the 
> previous two situations. The estimated inter-reliability of the' 
judges for Situation 3 was ,75, ^ ' ' . 

To determine if significant differences occurred between the 
two treatment groups, the judges' scores were again combined for 
each subject and a chi-square analysis was conducted,' The number 
r of local career education directors by total score and treatment 
group is presented in Table III-12, An analysis of the data re- 
vealed a chi-square value of 5,^99, The chi-square value needed 
for significance at the ,05 level (one-taiTed- test) with ID degrees 
of freedom was 18.3, Therefore, ^^Jj}ere was no statistical differ- 
ence between the two treatment groups ability to respond to Situ-' - 
ation 3, . ; 



TABLE III-IO i 
JUDGES ' RATINGS OF RESPONSES TO SITUATION 3 BY SUBJECT 

Subject's Score 



Subject 


Judge 1 


Judge 2 


Judge 


3 Total 


■ ■ .. l' 


3 


3 


4 


10 


-2 


3 


1 ' 




, 7 


3 


3 


.1 


2 


6 


A 


.5 


4 


1 


10 . 


5 


1 


1 


1 


■ ' 3 


6 


1 


1 


1 


3 


7 


2 


1 " 


2 


5 


8^ 


3 


1 


3 


7 


9" 


5 / , 


4 . 


5 - 


14 


10 


. 3 


3 


3 


* 9 


11 


5 . 


4 


5 


14 


12 


3 


1 


3 


7 


13 


1 


4/ 


1 .. 


6 


* 14 


3 


1 


1 


5 


15 


4 


3 


3 


10 


16 


4 


1 


2, ^ 


7 


17 


3 


3 


3 


9 


18 


3 


2 


3 


8 


19 


4 


5 


2 


11 


20 


4 


3 


5 


12 


21 


^4 


1 * 


2 


7 


22 


4 


4 


1 


9 


.23 ' 


5 


5 


A 


14 


24 


2 


1 


1 


4 


25 


1 


2 


• 2 . 


5 


26 


1 


1 . 


3 


5 


27 


3 


2 ■ 


- 2 


7 


28 * 


5 


'l 


3 


9 


29 


1 


1 


2 


4 


30 


2 


1 


, 2 


. . 5 


31 


4 


1 


>.2 


7 


32 


3 


1 


4 


8, 


33 


3 


1 


2 


6 


34 


4 


5 


5 


14 


35 


2 


5 


3 


10 


36 


2 


1 


4 


7 


37 


°3 


V 1 


3 




38 


1 


1 


2 




39 


2 


1 


3 


6 


40 


1 


1 


1 


3 


41 . 


2 


* 5 


2 


9 


42 


5 


- 2 


2 ■ 


9 


43 ■ 


2 


2 


1 


5 


44 


1 


1 


1 


3 


45 . 


2 


1 


4 


7 


^6 


5 


1 


5 


11 . 


ore 


2.89 


2.09 


2.59 





/ 
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32 



43 



TABLE III-ll 
SUMMARY ANALYSIS OF VARIANCE FOR SITUATION 3 



Source of Variation SS df MS ' 

Between people ~ 139.76 45 ■ 3.11 ■ 

\ Within people ' 120.67 138 [ -87 o A 

Between judges 15.17 2 I 7.59 ^ 

Residual. . 105.50 136' .78 

Total 260.43 183 



; . TABLE III-12 

^^UMBER OF LOCAL CAREER EDUCATION DIRECTORS BY TOTAL SCORE 
AND TREATMENT FOR SITUATION 3 



Total Score 






Number of Local 


Career Education Directors 


Treatment! 


Treatinent2 


Total 


■3 






2 


2 


^ 4 


4 






, 1 ; 


2 


3 








3. 


3 


6 








2 


2 


4 


7 , 






6 


4 


10 


8 






1 


1 


2 


,9 






3 


3 


6 


10 






2 


1 ' 


3 


11 






0 


3 


3 


12 






1 


0 


1 


14 






1 


3 


4 


Total 






•' 22' 


24 


46 
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CHAPTER iVv 

PRODUCT UTILIZATION 

a , ^, . , ■ ■ ^ . . •■ . '• 

Introduction 

Product utilization is ^ functional section of Thie Center 
which is responsible for developing and implementing plans which 
insure the use of designated products by the appropriate . 
audiences. To accpmplish this the product utilization organiza- 
tional structure consists 'Of a planning unit, a production 
unit (including typing, graphics, and audio-visual preparation, 
and duplicat;!Lon of materials) , a promotion unit, and a distribu- 
tion unit. An individual within the planning unit is designated 
as the representative to work with the director of a project 
in formulating and coordinating his/her needs for product 
utilization. This involvement by the product utilization 
planner begins at the inception of the project and incrementally 
increases as products are completed . 

In the case of the project reported here there were twq 
products:.^ Ojie was the handbook and the second was this report. 
Therefore^, plans were developed, coordinated, and are now being 
implemented to obtain an appropriate distribution of these two 
p:^oducts. This chapter explains what wAs done by the product 
utilization section and outlines the activities to be performed.j 

■ . V. . , . "■ • . . ■ . ^ 

There, were basically nine categories of activities in 
which pf^oduct utilization was involved. 

1^ Program monitoring 

2. Development of the program technical plan 

3. ^ Conduct of a market analysis , • 

4. ' Development of production and packaging specifications 



Development of promotion, distribution, and service 
specifications 

6. Preparation of a product utilization plan 

y 7. Conduct of promotional activities ^ 

8. Conduct of produci? production activities 

9» Conduct of product distribution arid service activities 
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Program Monitoring r ^ 

The product utilization planning representativie observed 
the program research and development activities, identified and 
recorded data concerning activities which would affect the 
ultimate disseminatibn of the project's products. Once recorded; 
this data was shared with the pro j ect director foi: confirmation 
and passed on to otfier units of product utilization^ when 
appropriate. 

■■ t . ' ' ' • . 

Monitoring was done by attending project staff meetings, 
accompanying project staff when they were talking with* test- 
site individuals , and being on' call for the purpose of coor- 
dinating general product production and promotion needs. 

Development of Techni cal Plan . 

The product utilization planner prepared the Product 
Utilization sections of the technical plans for program ac- 
tivities. This plan outlined procedures and tasks which were 
to be accomplished to move the products frlbm developers to * 
ultimate^ users . The results of this activity are further 
explained in latter activities. 

Conduct of Market Analysis . 

Purpose of the Market Survey 

ij ■ 

The intent of the market survey was to collect data to be 
used in the development of a product utilization plan for the 
Career Education Product Installation Handbook . Data were 
collected on the potential market for the handbook, perceived 
consumer benefits, consumer adoption behavior, peirceived 
consumer resistance, media by which potential consumers receive 
information about produG^t^, and distribution channels through 
which products reach the coifSumer. 

/ Objectives of the purvey . 

/ ■ ' . 

The market survey had the following specific objectives: 

• _ ' f 

1. To determine who^ are the potential users of the 
handbook (state directors and supervisors of vdca- a 
tional education, state coordinators of career educa- 
tion , superintendents , principals, local directors of 
career education) . 

2. To determine what factors cause consumer resistance 
to purcha^^ of the handbook (cost, difficult to use, 
impractic 
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3. To determine the appealing features of the handbook 
(ease of understanding, illustrative case studies,, 
appropriate language level). 

■t . 'ir 

4. To determine the kinds of career education; products 
being installed in public schools (curriculum materials) 

5. To determine how many copies of the handbook would be 
purchased at specific price levels ($3.00, $5.00, 

J $8.00). ^ . ' 

6. To determine who should receive promotional informa- 
tion about the handbook (state superintendents of 
public instruction, state coordinators of career 
edi^ation, superintendents, school boards, local 

^ directors of career education). ' ' * 

™^ — 'Methodology 

The Product Utilizati9n staff obtained the necessary market 
survey data through (1) telephone interviews with potential 
handbook users at the state and local level , and vocational 
teacher educators ; ( 2 ) from questions^, incorporated into the 
product usability instrumentpitioJi (field application) which 
was administered to 30 state and local sC^reer education directors 
and (3) from^. supplementary market survey information secured 
informally through the following activities: 

1. Potential user reaction to the CEPIH brochure and 
Centergram article. ^ 

2. Feedback received by Producjt^ Utilization staff members 
aii exhibits and displays at TheXTenter for Vocational 
Education and at professional meetings. 

3. Discussions between program staff and users during the 
field application of the handbook. 

Questions relating to determination of potential users 
of the handbook, factors causing consumer resistance, and de- 
termination of the appealing features of the handbook ' (objectives 
cpne, two, and three) were incorporated-into the evaluative 
instrumentation for the field application (see Appendix D) . 
'Thirty state and local career education directors were surveyed ' 
by questionnaire during field applicatioi> of the handbook at 
30 different sites. They were used for ^;termining the usability 
of the handbook because (1) they were mc*e familiar with the 
contents tt\an the telephone survey respondents , ( 2) they ^were 
exposed to the handbook over an extended period of ' time 
(6 months), and (3) they const±,tuted an audience perceived 
to be one of the primary potential user groups. H 
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A minimal telephone survey (20 calls) supplemented the 
above data , allowed contact of potential user.s not included in 
the field appligation, and provided an opportunity for two- 
way conununicatiori not possible with a paper ' survey . The sample 
for the telephone interview included educators at the state 
and local level as well as teacher educators. Participants were 
selected puxpdsively from the mailing list irtaintained by The 
Center with a^ttention directed to geographic location and area 
of interest (e.g,, agriculture, business and of f ice trade and 
industry, home economics) , The following sampling proportion 
was suggested: / : \, 



State Di^rectors of Vocational Education , 2 

State Career < Education Coordinators 4 

State Supervisors of Vocational Education 2 

Vocational Teacher Educators 4 

Local Superintendents and Principals 4 

'Local Curriculum Coordinators 4 

Total 20 



After 20 educators or school systems were identified as 
potential participants in a telephone interview, a letter was 

' sent to each indicating thaV The Center for Vocational Educa- 
tion was developing a Careep Education Product Installation 
Handjbook and was interested in assessing its receptivity and 
^potential usefulness. Each participant received a copy of the 
field test version of the handbook, for review. A follow-up 
telephone , interview using' a structure questionnaire was con- 
ducted by Product- Utilization staff to- assess their reactions 
to the handbook. Objectives one, two, four, five^ and six were 
addressed during the interview. A post card was enclosed with 
the initial letter upon which the potential participants ^ 
indicated their willingness to participate and provide the, 

"name, address, and telephone number of the individual to be 
contacted if other than themselves.. If nx> response, or negative 
responses werV received , additional names were to.be selected 
until 20 participants agreed to take part. Since the initial 
sample agreed to participate, it was not necessary to add names. 

Questionnaire * items for the field application were develope 
cooperatively by Product Utilization and program staff and the 
telephone interview instrument was prepared by Product Utiliza- 
tion staff. Instruments were foi;warded to the Evaluation 
Division of The Center for Vocational Education for .technical 
review. Once found to be technically sound, they were sient 
to The Center Protection of^ Human Subjects Committee for 
clearance. After appropriate revisions, the instruments were 
sent to NIE for final review and approval. 

■■ ^ ■ ■ V 
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Data Tabulation and Analyses; 

' ■ ■■ . . ■ / '■ , * ' . ' ^ • ' _ . 

Data securadLf rom the questionnaires completed by the 30 
state and local, career edu<;:ation directors and the information 
obtained through the 20 telephone interviews was tabulated and 
analyzed by, th.e Product Utilisation staff. Major findings were 
identified, ^conclusions drawn on the basis of these findings-, 
and recommendations formulated for the prpduct utilization plan. 
« ■ " •■. ' . • ^ , ' ■ • . ■ ' •* 

Return 

Out of 20 designated individuals, 10 -response^ were obtained 
in the telephone interview. This comprised a 50 percent 
response rate. . 

Findings 

In general, the ^ response v/as that the handbook is seen as 
a useful^tool for ^those involved witlV the implementation of 
career education. Also, there was indications from two or. 
three of the respondents that indicated the'y felt the handbpok 
had applicability in a broader domain of product and j^rcigrSun - * 
implementation processes. Several of the ^respondents saw the 
use of the handbook in inservice training of individuals that 
might be responsible for product and program implementation. 

. The respondents indicated that the * primary users of the - 
handbook would be curriculum developers, graduate teacher 
educators, and local supervisors of career education programs. 
A second set of users would be state department consultants or 
supervisors, and school administrators. ".The least likely to 
use the handbook would be teachers, researchers , and state 
department administrators. 

Only two out of the niixe resp9ndents in the telephone 
survey had anything they specifically indicated as objectionable 
about the handbook. The following are a listing of the comments 

by those individuals. ~' > ; ^ ' 

• \^ - • ■ 

Not attracted to "workbook" approach. It seems cumber some.. 

' ■ \^ ■ . ' : 
Football "jobks" turn lots of people off^v 

Too heavy on. process. ^ 

■ ■" . -'"^ . ■ / 

If r were impJLementing I would not be apt to do the 

"'workbook exercises. ** 

.- ■ , . . ' ■ _ . ' r ■• . 

None, but there are some technicalities such as/ fait 
accompli and threats o£ punishment which would not be 
appropriate in our - situation . . 

. . ' •• ■■ . ^ ^ ' ■ . * ■ 
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All Vhandbooks" tend to get left on the shelf • A coor- 
dinator needs to encourage use. 

•^^ --i^' V . •. . ^ - 

In terms of what individuals might expect to pay for the ♦ 
handbook the estimates ranged from $2.00 to $25.00. However, 
..the mp.&t .oftien quoted price /range was' from ^5,00 to $8\00. 

All telephone survey ^respondents indicated that they would 
purchase /at least one copy of the hai^dbook, . Three respondents ' 
indicated that they would probably purchase more than one copy,. 
Possibly 5 to 10, to— as many as. 35 copies. A couple of ^ 
respondents indicated that they would prefer to p'urchase one 
copy and duplicate it themselves if need^^be.. 

Promotion should to individuals such as: career educa- 
tion consultants at the state and local levels , currictilum 
developems and supervisors, RCU. directpre=f'==4:eacher educators, 
and professional development directors. Sojrre other persons 
would be guidance personnel and local voca<tional education 
directors • ' . ^ , 

The means of promotion should be multiple. Using brochures 
and flyer^ announcements in professional journals and news- 
letters, /and exhibits at professional conferences. 

' Con 

The handbook generally will BS^e 11 received,^ -However, 
there will be some reaction to the football faming analogy and 
possibly a label. Qf male sex bias toward the content. Hapefully^ 
these defects will be corrected itx the revised manuscript-1 
Encouragement. will be necessary in order for it to be used to 
its fullest potential Inseryice workshops and team approaches 
to the implementation and. use of the 4iandbook 3eem desirable. 
It will be easier to ideritify who will not see the handbook 
relevant to their professional responsibilitiea tharf to list 
^ those who wi,ll. Generally teachers, state administrators and 
researchers are ones who may find less application of the 
handbook to their immediate role responsibilities. Other 
educational professionals should find the information germain 
to their" roles^w 

clistomers would expect to pay somewhere between five to 
eight dollars for the book. Under present rates. The Center 
cost would be approximately $11.50 which is probably not too 
^far' above the upper limit , of the expectation . 
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In^iuranary, it can be stated that the handbook has utility V 
from the consumers ' viewpoint and they 'will purchase copies 
and. use them in their; career education program implementation. 
PrcMuotion should be as broad and varied as possible de:- > 
emphasising only local teachers , state administrators and ' ^ 
researchers. 

■ ■ ^ ■■ . ■ . > ■ ■ . 

Development of Production and Packaging Specifications 

> The product utilization section asisisted program staff 
in the determination of format and packaging considerations. 
One of the major issues which arose early around this area of 
concern was whether there should be one, or two, or possibly V 
even other versions of the .handbook . The question ^hinged around 
whether advocates of career education at various levels (e.g., 
state, regional, local) wauld need different guidelines. The 
technical plan set the over^I issue in the following statement: 

The decision to produce more than one version of . 
the CEPIH must be based on a significant ampunt 
of evidence showing real differences in the roles 
of career education product advocates proposed as - 
users of this handbook. ' , 

' ' . ^ • ■" • • 

Through discussions with the program steering committee, 

career education project directors and others,, it was determii>ed 

that only one, version was needed.' The ^handbook is primarily 

designed for local educational personnel in installing or. 

implementing career education related products. It was the 

general consensus of those*- with whonj we talked that these 

people did not greatly differ in their needs and concerns 

related to implementing career education related products. 

Development of Promotion, Distr^ibution, and Service Specifications 

- ■ ■ .■ ■ ' ■ 1., 

This- was a planning task which preceded the preparation 
of the product utilization plan. The respective sections 
of product utilization provided input to the accomplishmentry 
of this task . 

Preparation of a Product Utilization Plan 

The produc-t^utilization planner had primary responsibility 
for de^/eloping the product utili zation ^plan. It provided a 
guide tor the production, promotion , and\distribution of the 
products of the program on a cost-recovery basis . 




Conduct of Production, Promotion ^ Distribution/ » . : , 

and S ervice Activities ^ ' - 

— ; \ ' ■ ■ 

'"Production and proitlbtion :s^ryices Were •conducted through ' 
the direction of the program. Distribution activities are 
to. 'be .performed when the products are completed and ready for 
sale on a cost-recovery basis, -j ^ ' ' 



.41 



. ceo 



ERIC 



CHAPTER 
SUMMARY 



Findings/ 
\ 1. 



2. 



3. 



4. 



The summative evaluation participants using the^CEPIH 
scored significantly ^different on^ the pre- and post- 
test of career education product installation knowledg 
than those using the comparison book. / \ ; \, 

The summatiye .^valuatilDn participants using the CEPIH 
gained: more kndlwedgQ -^hah .the participants using the 
comparison book. \ * ^ 



sxumnative evaluation participants V. experience as, 
ject directors had no i^ignif icant association with 



The 

pro^ec-c airecrors naa no signj 
their cognitive knowledge acquisition. 



The summative evaluation participants' mean pretest 
score., was signlf icantl-y different from the mean post-* 
^ test score on cognitive knowledge of career education 

■installation procedures. 

,5 . , Thd^ suimnative evaluation participant-s \ strategies fpV 
resolving e^ch of . the .simulated carfeer^ education y 
installation problems did not differ significantly 
by treatirient group. , 

Claims for the Handbook / ^ 

1. The content of the handbook is based on reliable 
information.' , 



2. The handbook is relevant for career education 
implementation problems.* ;^ ^ • 

3. The material in the handboo}t-4s free of serious social 
(sexual , ethnic , etc . ) l>ias6s. 

4. The handbook, is easy to use; 

5. 'The handbook is inexpensive. 

6. Use of the handbook will improve a care«; jeducation 
director 's ability to/ f drmula^te an effeKive ' " 
implementaJl^ion ^strategy . * . o' * jf 
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7. The handb'ook is as eff'Sctive as currently available 
material? in assisting career education project 
directors to formulate implementation strategies. . 

Recoimnendations . , ' 

■ , . ' • ■' ' -, . V ' "■ ' * 

^ - 1. The handbook- should be made available to career educa- 
tion project directors in local education agencieis. 

2., Potential users of the handbook should be introduced 
to the implementation process via small group in- 
'service workshops. 

need career education project directors may be 
ffectivetly use the booklet of tactics; but, 
, the use of the procedural guide with the 
tacties is advised. ■ / 
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APPENDIX A. , ' 

PERSONS CONTRIBUTING TO THE DEVELOPMENT OF THE HANDBOOK 



The Steering Committee 

Dr. Elvis H. Arterbury 
Project Director 
Partnerjs in Career 

Education 
Suite 130 ^ . 

1201 N. Watson Road 
Arlington, Texas 7 6011 



Dr . Kenneth ■ Eaddy 
Room 2 58 
Knott Building 
Tallahassee, Flof-ida 32304 

Dr. David L. Jesser 
Career Educatioja Project 

Director 
Chiet^tkte School Officer 
,1201 16th^ Street NW 
WashiTigton, D. C. 20036 

The Revision Commititee 

Mrs. Mary Anna Elam,- 
Director of Career ; 

Developmerft 
Mad River - Green Local 

Schools ' 
3920 Fairfield. Pike 
Springfield/ Ohio 45502 

Mr. Robert Megow, Director 
Orange County Career 

Development Program 
410 Woods Street 
Orlando, Florida , 32805 



3. Formative Evaluatidn Field 
Test Sites' ' " 

Ms. bbrothy Riley 
Career Education , 
^ Coordinator 
Mansfield City Schools ^ ^ 
270' West Sixth Street 
: Mansfield, Ohio :^44052 

.< • ■ 

Dr. Marvin. Rasmussen 
Director of Career 

„ Education . \ ^ 
Portland Public Schod\ 
District > 
631 NE Clackamas Street 
Portland, Oregon 97208 

Dr. ^Kenneth Tye 
IDEA Research Division 
110 0 GlejQdon Avenue 

Suite 950 
University of Califoi^nia 
Los Angeles, California - 
90024 

Dr. Ralph Wileman 
Associate Professor of 

Education 
University of North 

Carolina at Chapel Hill 
Chapel Hill, North Carolina 
27514 



3. 



Formative Evaluation Field Test Sites 



(Cont'd) 



Dr. Charles Mojokowski . 
. Coordinator-Support * 
Services 
Department of Education 
Hayes Street 
. Providence, Rhode Island 
0*2908 

■ * ' 

Mr.^TDbn Taylor 

Career -Education Center 

1003 West Cannon 

Ft.~ Worthy, Texas 76104 

Mr. DeRoy Gorham 
Coordinator of Career 

Development 
'Lorain City School 

District 
Brownell and Tenth 

Streets • 
Lorain, Ohio- 4 40 52 

Mr. Jack Ford 
Room &09 ' 

State Office Building 
65 South Front Street 
Columbus, Ohio 4 3215 

Mrs. Andrea Barrett 
Career Education Specialist 
Wakulla County Schools ' 
P.O. Box 98 

Crawf ordville , Florida 
32327 ; / 

Mr. Clif foi^'d ' A. Bellum - 
Asst. Super.; 
Vocational, Technical and 

Adult Education 
Sarasota County Public 

Schools 
2418 Hatton Street 
Sarasota, Florida 33577 



,Pr. Kenneth Eaddy, Chief 
Bureau of Vocational Research : 

and Education ' • 
Florida Department of Education 
Room 258, Knot't Building 
Tallahassee-, Flprida , 3^04 <r 

Mr. D/ Elmore, Director 
Career Education Program 
Okaloosa X^ounty Public Schools 
201 Marilyn Avenue 
Fort Walton Beach, Florida 
32548 ^ 

Mrs. Myrtle Hunt, Director 
Pinellas County Career 
Education Project • 
3230 Nifith Avenue, South - 
St. Petersburg, Florida 33711 

Mrs. Brenda Dykes . 
Career Education Coord iitariror^''^^ 
/Sulphur Springs Indiana School 
> District 
P.O.' Box 276 ' 
Sulphur Springs, Texas 75482 

Thomas R. Forrest . 
Coordinator of Career 

^ Education 
Harlingen Consolidated. 

Independent School District 
Jiarrison Street , 
Harlingen , Texas 78 550 

Mrs. Mary Anna Elam 
Career Development Program 
Mad River Green Local School 
3920 Fairfield Pike 
Springfield, Ohio 45502 
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Formative Evaluation Field Test Sites ( Cont'd) 



Dr, Leonard JAckson 

Director M 
Career Educa^ft>n . 
Alachua "Coun^ Public 

Schools 
25 S. E. Second Place 
Gaines\/ille> Florida 32601 

Mr. 'Robert S. Megow 

Director ^ 
Prange County Career 

Education Project. 
4ia ^pods Street - 
.Orlando, Florida 32805 

Mr. Michael Neal 

Director 
Brevard K~14 Career 

Education Pro'ject 
Cocoa High SghooL" . 
2000 Rosetine Street 
Cocoa, Florida' J2922 

"Dr. James Smith, Director , 
Broward Couiity Career 

Education Project 
1001 Northwest Fourth 4. 

Street 
Ft. Lauderdale, Florida 

33310 



Mr, James C, Talley 

Director 
Jjeon. County Career 

Education Project 
925 Miccousukee Road 
Tallahassee , Florida 

Harold Lichtenwald 
Director of Carreer 

Education 
Edison Middle School 
Pallas, Texas 75204 



32303 



VDr. Joe A. Vicars ^ 
. pirector , ^Career D^elopment 
and Contirtuj.ng Education 
Austin Independent School 

bistirilpt . 
6100 Guadalupe 
; Axisti.n, T^^xas 78X52 

Mrs. F^ay- oWens (Mary Sue) 
* Career Education Director 
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APPENDIXB ' 

V ^. • ■ • ■•• , 

. SUMMATIVE' EVALUATION MATERIALS . 
. ; : 1. Workshop Agenda- 

CAREER EDOCATION PRODUCT INSTALLATION ' 
• Simulation Workshop Agenda 

- . ' ' > Day 1 . . 

1:30 - 2:00 , Introdtiction, Purposes, and Obje'ctives 

2:00 - 3:00 Pretest 

3:00 - 3:15 Coffee Break ' 

3; 15; - 5:GfO Distribute Workshop Materials - Re^d and 

^ become acquainted with References 

■ ■ * ■ ^ . ■ :• 

Day 2 



8: 30 - 9 : 30 ^ . situation #1 ' - < 

9:30 - -10:30 Small Group Discussion of Situation al 

^ Group A " ^ ■ ' " • 

Group B 

Group C 

10: 30 - 11: 30 Situation #2 

11: 30 - 1:00"^ / Lunch 
1:00 - 2:00 Small Group Discussion of Situation #2 

Group A " ^ . ' 
■ Group B * 
Group C # ' 

2:00 - 3:00 Situation #3 - 

3:00 - 3:15 ^ doffee Breaks ^ - ' 

.3:15 - 4:15 Small Group Discussion of Situation #3 

Group A 

" / / Group B . 

Group C . ' 

4:15 - 5:00 Group Discussion of the Three Situations 

. ' - pay 3 . ' . ^ 

8:30 ^ 9:00 ^ Closing Remarks ' ' 
9:00 - 10:0.0 Post-Test " 

10:00 ^10:15 CtDffee Break ~ / 

•in'lc ^ J-?-*"^^" Career Education Product Installation Handbook 

10:45 11:30 J^eimbur sement Procedure" 

Hi 30 Adjourn . - . , - . 
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2/: Cognitive Test * . 1 

CAREER-.EDUCATION PRODUGT INSTALLS TEST 



) 



DIRECTIONS: The Career Education :Product Installatxbn, Test is 
'.designed to determine your knowledge of and experierxqe in instal- 
. lation strategies. Your .re'sponse ' will be used in det:^rmining 
the effectiveness d£ career » education materials in d,eiye loping' 
strategies for implementation. The information you ij^e will not 
be associated withi your naihe in reporting the 'data, ^our 
responses will be used in the analysis of group data iginly. 
Participation in this; study is strictly voluntary. /The individ- 
ual information and^bores will not be ma,de 'available tq any 
. individual^ organiz^io^, or agency. Your individual' reactions 
wiir be destroyed by'^OQtober 31^ 1975. We appreciate and value 
your professional reatctions. ' ) 

Please read each question carefully.^ . including each of 'the 
responses. Then darken in the letter* on your ANSWER SHEET-that 
corresponds with what you think is the one bes^t answer. Try to 
answer all of the questions. The following example question 
should help you to unde|*stand exactly what you; are to do: 

About how. long is a normal work .week?. 

A. 40 hours 

> B. 30 hours 

C . 25 hours ' . 

. ' P. 60 hours 

For this question you would haive darkened in the A on your 
ANSWER SHEET^ since a work week is normally 40 hours Xong. 

Please make sure your name is Iprinted in the appropriate place 
.on the ANSWER SHEET. V : ^ ' 

1. Which of the following is least essential^ to the establish- 
*■ ment of incremental objectives? 

- . " r'- ' 

a. Selection of short-term ^behaviors which are critical to 
the success of the instillation .activity. • / 

b. Determination of conditifohs in the client setting which 
may influence the achievlement of the objective. 

' c. . Selection of comprehensible, specific behaviors." 

d. Deteintiination of . observable criteria for evaluating ' ■ 
achievement of the: objective. / 



' : ' - ' ' , i/ 

Which method of collecting evaluative information on the 
impact of installation ^.tactics yields the most' objective - 
data? ^ ^ i ' 

observation of antecedent co^itions 

b. personal interviews with the project director 

c. personal interviews with' an evaluator associated with . 
^ the project ' . 

^ d. perso.nal interviews with ait evaluator not^ associated 
with' the ^{;)roject 

■ i' . ' - * ^ ■ . " 

*^ 

•The tactic of competition has as one of its goals the ' ^ 
generation of: 

■ • . ■• • V . ■ . / ^ " ' 

a. , least'-cost solutions to problems 

b. interest in the product 

the elimination of alternative solutions 

d. all of the above : j 

'Although a consensus is seldom reached, one way to gain 
infjormation from teachers when solving problems is: 

a. to observe their behavior . 

b. brainstorming 

c. coli*aborative activity 

d. small group discussion , 

/ ' 

During negotiation periods, it is advisable to overstate 
one's demands as this allows for possible: 

a, brainstorming 

b. consensus . 

cJ corp^iromise ^ - - / 

d. Ji(±\ of the above / 



If a project manager wanted to include specific teachers in 
the development and revision of m&terials, whnt, tactic 
would be chosen? ^ . . . 

a. role playing • 
b consultation . 

c. staff develooment 

d. small. gVoup discussion 
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If an individual resists innovation, a pro ject manager might 
try: ^ * ^ ■ . 

a, mass media communication 
brainstorming 

c. a collabora:tive activity 

d. setting a deadline fo;r using the innovation 



8, A pilot test of a product involves: ^ 

^ a. demonstrating a product on site 

b. trying the product on a limited basis 

c . f aci litating communication between school and community^ 
^3. endorsement by credible users of the product 



Installation tactics a-re "designed to; 



a. work- in all situations 

b. achieve incremental objectives 

c. gain wide audience parti'cipation 
*d, all of the above 
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The . heterogeqity of the teaching' staff would be a character- 
istic of which of the following: 



a. tactic 

b. product 

c. client 

d. advocate 





Research has shown that: 



a. affluent school districts tend to be more innovative ) 
h* poor school districts tend to resist innovation 

c. the relationship between affluence and resistance to 
innovation is not clear 

d. affluent school districts with small school populations- 
are more resistant to innovation 
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Which of the following provides, thei best means of extending 
advocacy of a career education product during the evaluation 
stage of installation? • 

a. Interpersonal relationships among clients 

b. Mass media « 

c. Administrative order .to use the product 

d. Promotional information - 



An ins ta nation obj ective should be : 

a. relatively attainable 

b. restricted to a time table " 

c. operational 

d. all of the above 



The initial step in formulating an installation strategy 
is the: 

a. profiling of influential elements 

b. develbpment of an evaluation plan 

c. establishment of incremental objectives 
cK selection of appropriate tactics 



Assume three elements are present in product instali§,tion : 
the product , the client, and the advocate. Which of these 
interact? 

a. product and client 

b. product and advocate 

c. client and advocate ^ 

d. all of the above 



In order to facilitate product installation, one should 
assume that : 

a. a good product does not need an advocate 

b. ^ use of a product is based on its merits 

c. people make rational decisions ^ 

d. none of the above 
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!?• A career education advoc4k*^'s goal is to: 

a, design a product to meet some curriculum objective - . 

^b. get a career education product accepted and/or used 

c. use and evaluate a product 

d. identify needs for change in the education system 

18. A tactic is: 

a, a method of evaluating product efficiency 

b. usually initiated by a client 

..c. usually the result of the advocate's personality 

d, an action taken to achieve insiallation 

( : . ■ ■ * ■ 

19. When a series of Evaluations continue to show general 
acceptance on the part of the individuals being influenced, 

a. time should not be used for further evaluation 

b. evaluations should bfe made aft?er each new activity is 
initiated . . ' ^ 

c. it is best to let the career education product sell 
itself 

d. deviation from the installation strategy should take the 
form , of earlier deadlines 

20. In evaluating the installation of a career education product, 
the primary concern of the evaluator is the 

a. acceptance of the project director 

b • consequences, of unanticipated events 

c, extent to which goals werd met 

d. cost-effectiveness of the evaluation procedures 

21. Evaluation activities cannot guarantee that a final objective 
will be met because 

a. the. timing and sequencing of installation activities 
should be flexible 

b. * the evaluator can only make recommendations 

c. most methods of evaluation lack of objectivity 

d. all of the above 
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When initiating the activiti^g of an installation strategy, 
an effective career education pr^^ect director will proceed 




a. with cauti°n 

b . with haste ' 

c. with- firm, uncha^igeable objectives 

d. without deviating ^^om tj^g schedule of activities 

f . ■ " ■ ' 

A change agent can t>e more forceful in attempting to imple- 
ment a career education product when 

a. the agent is viewed as an "outsider" 

b. the agent tias been with the organization for a long time 
■c. the agent has technical information about 'the, product 

d. the agent, is resPected by the users 

It may be necessary for a ch^j^gg advocate- |<r5',;^ain the 
confidence of a comifumty "option leader v^.t|.pg career 
education product I'- , ' 

a. is not oriented to students ' 

b. is likely to compete with resources in the existing 
program 

c. is going to i:eq^^re exti:en,e reorganization before 
implementation 

d. is associated with values that have, a hig^L^tential for 
organized resistance 

A change agent should emphasigg various important character- 
istics of a career education p^-oduct because 

a. different audiences may b© interested in^'dif f grent 
aspects of the product 

b. different school districts are differentially disposed 
. toward educational ^^nov^tion 

c. teachers- v^i^y ^'^ their abnity to use products 

d. all products are not alTiKg 
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Individuals who are introduced to a career education product 
will go through various stages of accepting that product. 
To facilitate this acceptance^ a change agent should 



a • insist on all potential users observing the pr'oduct in 
action as soon as possible 

b, establish separate objectives for each state of adoption 

c, persuade the individuals to give the product a trial run 

d, evaluate the product 



To measure whether an objective has be^n attained requires 
that it be y/^^^^^ 



Which of the following is least true? 

a. Products should be field tested before installation, 

b. "Change advocates should be sensitive to other peopl.e. 

c. Good products sell themselves . 

d. Volunteers may be used to try out products. 



Coercive tactics tend to be most effective when 



a. progress oh the installation of the innovation is 
deadlocked . 

b. the installation activity has just begun 

c, other solutions to the probleiji have been exhausted 

d, the advocate^ is . in a superordinate position to the 
person being influenced 



Most installation strategies in career education 

a, require informative tactics ^ - . - 

b- require persuasive tactics 

c. require coercive tactics 

d, require a mix of the above types of tactics 




c . 



a. 



operational (behaviorally defined) 
attainable within a certain time limit 
independent of other ob j ectives 
all of the above ^ 




Which of the following most accurately describes the use of 
installation tactics? 

a. All tactics can be- used equally well by any advocate. 

b. Installation tactics have unique uses with people and 
places . 

c. Most tactics can be used with confidence. 

d. The selection and use of most tactics is a trial .^nd 
error process. 

Which of the following tends to most influence the choice 
of ah installation tactic? 

a. The personal preferences of an advocate. 

b. The location of the career education project director ' s 
-^6??ice. 

c. The purpose for which the tactic is to be used. 

d. The consequences of using the tactic. ' 

One reason why installation of a career education product 
may stall is that too many ar^jfcernative activitiea^ present 
themselves at a given point in time. To avoid this problem, 

a. plan an overall strategy according to a timeVtable 

b. select only those tactics which you know to bp effective 

c. use the least coercive activities 

d. plan for discussion groups whenever that probiem arises 

The attitudes of which pf the following groups must b^ 
considered in planning against organized resistance? 

a. teachers . * 

b. principals / 

c. superintendents 

d. all of the above 



The characteristic of a career education product^ which is 
least important to the change advox:ate is 

a. how the product is oriented to the labor market 

b. where the produbt was developed 

c. the cost of the product 

d. all of the above are equally import^t 
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36. The _ actions initiated for the installation of career edu- 
cation products 

a. are generally product specific 

b. are^not specif ic to the stage of adoption 

c. ^are ^imed at gaining support for the product 

d. are general ^nough^ tp be used with any client ' 

37. Before selecting an 'installation activity,- the proiect 
• . director should profile . • 

^a. his or her position as a product advocate 

b. the utility of the career education product 

. c. the potential user's perception of the product 

all of the abbvr ^ ^ 
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When contacting persons for the first time, career education 
advocates should avoid which of the following: 

a. a salesman image ' ' , 

b. a low profile 

c. ^ establishing a clincial rela4:ionship • 

d. giving them technical information abc3ut the product 

Aa underdeveloped career education product 

. - a.^. should never be installed into a school system 

>. may be installed successfully in large school districts 

c. may be helpTul to beginning teachers 

d. may be used successfully by resourceful teachers 

40. The relationship between career education product users and 
their supervisors is s^ch that . 

a. endorsement of a product, by supervisors will insure 
continued use of the product 

b. supervisors will withhold judgement Ijf a product until 
the users have evaluated it 

c. users fKequdntly suspend use of an "approved" product 
; when unsupervised 

d. the opinion of users is general^ly inconsistent with that 
, of supervisors 
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41. The effective timing of an -installation tactic depends upon- 
many factors. Which of the following is the most important? 

the readiness of the client fSt the product 
the readiness of the advocate to plead a case for the 
product V ' 

, c.^ the stage of development of the product 

d. recent past eve-ni^ which affect the ability of the school 
district to pay for the product 

42. An initial contact with a client for the purpose of estab- 
lishing the role of the change advocate with a single 
individual is^best accomplished with 

a. a visit from the change advocate 

b« a telephone call 

c. a personal letter . i \ ■ 

d. an informative brochure ^ ^ ^ 

4 3. ' When interacting with ^her people ,Van_^vocate should 

a. check to see what worked in similar situations 

b. carry^ out product installation plans regardless of the 
client's response I 

^ - tC ask questionsc^of the client' to determine which installa- 
tion tactic is most likely to succeed 
d. be guided .by his or her knowledge of what is needed -in 
-the school district 



44. When initiating career education activities, it is most 
important to 

a. have the approval of the administration 

b. have the goo^ill of the teachers 

c. plan the schedule of activities- in advance 

d. have a good product 
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When^ fe?ormulati^ng an installation tactic, it is best to 

a. consider the interests and desires of tha client audience 

b. look at^^^he impact of the previous installatiqn tactic on' 
the client j 

c. rely on your own intuition^ f 

d^, ask t?he building principal for permission: to involve 
teachers in career education activities * 
' ' ■ ■ ■ ' ' \ 

When reformulating an installation strategy, career education 
advocates should : 

a. use tactics which are high" pressure 

b. look for unanticipated responses of clients to 
tactics 

c. not change the incremental objective 

d. base your decisions on empirical data only 
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One way to improve the chances of achieving your" installation 
objectives when reformulating diffusion tactics is to 

a. use more coercive tactics with clients . h 

b. modify the product " 

c. diagnose more carefully the client's perceived need for 
the product ^ . . ^ 

d. adjust the objectives downward 

■ ' ■ \z — i ■ ' ■ 

The most important item to consider wh!en diagnosing the need 
far r^w installation tac|tics is to detjermine ) 

,a. what tactics are most likely to be acceptable to the 
client ! 

b. which tactics are most likely to get' the job done 
'c. why the previously used ;tactics failed ^ 

d, who was responsible for the failurje'^ ' 

Which one of the following statements is not true: 

a. The selection of an installation tactile is a personal 

decision. ' ^ , • 

b • Good products sell themselves . ' 

c. Evaluation is a continuous process. 

d. Objectives may be incremental in nature, c? 



66 77 



-12- 



The periodical assyessment of the impact of an installation 
strategy may provide the project director with a -Joasis for 

a . reformulating the installation strategy 

b. reinforcing support for the installation of a career 
education product , 

c. describing the program to individuals interested in its 
adoption 

d . all of the above * 
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3. Simulated Situations . 
CAREER EDUCATION . PRODUCT INSTALLATION TEST 



DIRECTIONS: The Career Education Product Installation Test is 
lati?rstratea-:r*"v knowledge of and experience in ins-tal- 

^ ^ • .^"""^ response will be used in determining 
the effectiveness of career education materials in developing 
strategies for implementation. The information you. use will nat 
be associated with your name in rep6rting the data. Your " 
responses^will be used.in the analysis of group data only. : 
Participation in this study is strictly voluntary. The individ- " 
ual information and scores will net be maS available to any ' 
individu-al, organization, or agency. Your individual reactions 
will be destroyed by October 31, 1975. We appreciate Jnd va?Se 
your professional reactions. ^ vaiue 

Please read^ the situation carefully. Based on your understanding 

oroh^;/''^v^"^' ?'^^"^ P^^?^^^ ^ P^^" C5f^ action for solving the ^ 
problem. , Your plan of action should include,the following steps: 

1. What-' other conditions and/or information are you assuming 
in responding to this situation? 

2. Describe -in specific terms, the problem ^o be resolved - 

3. Identify potential tactics for use in dealing with this 
problem. ' 

4..- List, the advantages and disadvantages of each potential 
tactic . 

5. select, in your opinion, the one best tactic for dealing, 
with the problem, » 

6. Indicate how -you will, determine the effectiveness" of the 
tactic ^in dealing with the problem. 

7. List potential consec^uences of the tactic that the local 
_^pro]ect director could expect. 

I Prepare your written response on the attacheH ANSWER SHEET. 



Situation 1 



y^u have Dust. been employed by a school system to plan and 
implement a career education program throughout the district 
The school district includes both metropolitan and suburban 
communities with school attendance centers in each co^unity 

IT^^L^""^ h''h° ^r^^^^^' 70 elementary school attendance centers,. 
14 Dunior high schools, and .6 senior high schools' in .the district 
The school district contains businesses from all industrial a^d 
professional areas.- .There has been ' relatively little contact' 
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with career education up to this time. The local citizen seems 
to have a high level of interest in the school district.' In fact 
a previous^ special project conducted- by the school wag criticized 
by the local citizens as being non-essential and frivolous. 
What tactics will you utilize in planning the career education 
program? 



A, 



■■a ' 
•I 
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CAREER EDUCATI0NnPRi6I)UCT INSTALLATION TEST 

■ 

■-'I • ■ 

A' ■ , . 

Situation 1 • 



What other conditions and/or information are you assuming. in 
responding to this ^ituaitioii? ; ' 




Describe in specific terms the problem to be resolved, 



Identify potential tactics, for use in dealing with this 
problem • 



Listr the .advantages and disadvantages of each potential* tactic 



Select, in your opinion, the one best' t'actie for dealing with- * 
the problem, ' \ . ' . ' 




t ■ i . ■ .. . ^ ■ ■ : . . ■ 

■ * . At-- 

Indicate how you will" cJetermine the effectiveness of - the tac:tic 
in dealing With the problem, . • 



A. 




Situation 2 

At the end of the first year, you begaa-^to compare the progress 
of teachers in im^ementing career education activities in 
their classrobm teltching. A group of approximately 25 percent 
of the teachers have totally revised their course of study, ^ 

-course objectives and content, and instructional approaches.* 
They have based them on student interests , aptitudes, "and 

^abilities as expressed by their students and the literature . 
available describing general interests of students in the age 
group they were teaching. These teachers have skillfully 
combined career education concepts and activities , subject 
matter, and student interests and .problems to form the content 
of their courses. Another group of teaohers have not revised 
their program of study, are /teaching cat^r education-^ activities 
as separat^j units of instrudtion, and A^e pAaqing major emphasis 
on mastery of the ^subject matter as siiirfi^^ tnat the learner 
will assimilate all' .that is t>^ght and/ appjky it as needed to 
solve his or her problems. Se\?fen4:5><Els^^ of the teachers 

fall'lnto^'this group. What tactic will you'use in dealing wifely 
this situation? * ' r ■ 

■ : ' ■ ■ •• ' ■ ^ • 'L ■ 

c ^ Situation 3 ; , , 

The career educatibri program activities you are coordinating in 

the-^.school syste^isi^en underway for two and pne-half years*. 

As coqrdinator, you have succeeded, in (1) providing relea^^ime 

^^or-your teachers to plan career education activities at the . ' 
district's expanse, and ( 2) » establishing a sizeable budget^ 
to ..sujoport career education activities throughout the school 
system.'v The ^;iocai citizens have formed a "League of Taxpayers" 
to proterst higher taxes to support the local sphool program. 
The bpard of> education will be reviewing the progress of all 
programs .in the school to establish program priorities and ■ : 
budgets "^or the upcoming school year. Youvhave kept the board 
fully ii\formed of your career education program activities 
throughout the"" p^s.t two and one-half yeaijs . However, the costs 

.associated with the career education have' risen due to infla- 
tion and increased- activities of the teachers . What tactic > 
will you use with this problem. 




APPENDIX C - 

. DEVELOPMENT MATERIALS ~ 

1. SUMMARY OF THE April 25-26, 1974 MEETING 
^ WITH THE STEERING COMMITTEE 

■ . . ' . , ' ' c ^ ^ 

The Steering Cominittee (see the attached list for the names ' 
of the member assisting program staff in arriving at the 
following decisions: 

Endorsements * , 

1. The program should plan one version o^j^j^he handbook for use 
/in the field sites. This will minimizWdistribution problems 
and al^ow resources to be directed toward a quality product. 

2- The products used to further develop the handbook shoS^d 
be sufficiently tangible to allow the change advocate to 
determine ^ if it has been installed. Curriculum materials 
are perceived to be most appropriate for this use / 'with 
most of the attention being given to the exploration of 
careers during grades 6, 7, 8 and 9. 

3- The handbook is not a device for se;lecting career education 
products. When the handbook is used 'to devise* strategies , 

a product will have been identified for- installation in 
a particular site and a person will "have been given the 
responsibility for. facilitating the installation process . 

4, The handbook should be developed independent of any need " 
to provide technical assistance in its use. 

5. The handbook should be orientated to the process of 
installing career education products a'ncja not specific 

to the particular career education product being" installed . 

" ■' ^ ' 

Users of the handbook can be classified into two gross 
categories depending* upon the :^nction and size" of the 
agency: (1) a HUB agency such as a regional service 
center, a large stJhool/.distridtv a teacher education agency, 

^ state education agency (^hese -agencies would pxovide 
resgarch, development anc| evaluation .support to others . * 
, engaged in the process of education) ;'^and (2) a SCHOOL ' 
UNIT. suc2h "^s a small school district; a school, or even 
*a classroom which is primarily concerned with the learn/ 
teaching process. 

'• ^ • ' • / 73 

■■/ ■ . ■ 



The potential target audiences for ' the handbook were 
identified as: regional coordinators, local directors, 
principals, curriculum vice principals, career educators, 
supervisors, coordinators, state directors of vocational • 
education, and state advisory council members'. (Chief 
state school officers are too busy to take action on use 
of the handbook, but thfey should be kept' informed of its 
development.) 

J' 

Handbook Specifications - 

The handbook should 'begin with general guidelines followed 
by more specific proceduj(pL activities whiqh will be 
-referred to by the user on a question by question basis. 
(Teacher users of the haiidbook are more likely to require 
more specific information about the installation of the 
product . than administratbrs of HUB personnel.) - 

Career i^ducation should rjot be defined' in the handbook.. 

Ttte handbook should b6 indexed in a format which will 
allow easy access to the information in the handbook; 
this information should b,e relatively "self-contained" 
within the sections of the handbook and be designed to 
require a minimum amount of reading for th6 desired 
information. - , 

Provide space- in the handbook for users to writo? the goals 
of the product being . installed. ' % 

Criteria should be developed to allow the users of the 
.h,andbook to evaluate characteristics of the product toeing 
installed, and to assess the ability of the. target audience 
setting to install the product effectively. 

A variety of suggested means for assessing the career 
education product and, the target audience setting should be 
included in the handbook. 

. . • * ' ■ . ' J. . ^ 

Additional Suggestions - . , 

The handbook should be installed in the development sites 
via inservice education of career education advocates. 
This will require the handbook be ready 'for distribution ' 
at- least by August 15, 1974: 

A conference phone call among selected development sites 
should be used to colle9t data at least midway in the field 
site effort. The four respondents should be made aware 
of the agenda in advance and be prepared to discuss the 
questions . . • 



The handbook should value the consumer of the career educa- 
tion product in a high" priority way* The handbook should 
indicate a belief that the. teacher (or student) can subvert 
the intent of product installation if they do not believe 
in it, , 

The word "power"] or "change" should not be used in the hand 
book. Rather the words " improvement" and "legitimization" 
should be used. «r 



SUMMARY OF THE JVIAY 9, 1974 MEETING WITH RALPH WILEMAN 

' > ' \ '■ ' '■ ^ ' . .. • • / 

Ralph Wileman spent May 9, 1974 consulting with program staff 
on the question of the Career Education Product Installation 
Handbook format and design. Several fundamental considera- 
tions were' confirmed: 

1. • The handbook should be a "how-to-do-it" b©ok for^ 

diffusion agents faced with the action to take in a ' 
specific setting. ' ^ ■■ 

2. We recognized the problem of users' varying levelS of , 
expertise and skill in making judgments about' the career 

r'educafc^ion product, and the audience for whom th\e product 
is intended. Therefore, :a' section with a narrative is 
planned which will describe the* use of specifi%jlr 
tactics int specific situations. This should have the * 
effect of humanizing the checklists which a;re ^ecessafy"^ 
to assure consideration of relevant variables . y ' 

'» 

3. The handbook format should allow users with varyihg 
degrees of experience to uiSe thet handbook equally 
well, e.g* / each section should^ be, written * in a some- 
what self-contained manner H^^wever , the sections 
wil*i be written at different levels of; sophistication, 
thus, allowing the user , to -:g:e£ef to examples or other 
information which explaix]s^ cQnceptjs in the handbook. 

4. The handbook shouldcontain a list of definitions 
including terms such as "installation ," "product , " 

etc . ■ _ ' . . ; 

^^^^^ ' " ' 

he handbook should be active not, passive. That is, • 

it should allow the user to^ formulate strategies. ' . 

This suggests the use of looseleaf pages which can be 

resupplied. ' A planninig sheet should ^be provided for 

each step. , ^ . ' 




6. 



A fold out could be used to keep th6 "model" for 
fo.rmulating strategies, in the mind' of th^ user . This 
schema/ also could be us^d for indexing the handbook. 



7, • The index of the handt)Ook should allow the user to 

classify his problem quickly and easily and find 
information which will aid in formulating actions to 
resolve the discrepancy^'. It is likely the handbook * 
* will cojitain more information than any one individual in 
a given situation would- use. This i^ ..why the index 
is so important, ■ - . / ' 

8. The index, should contain ripper erit words for users' 
problems. This. meaAs a nai^ative or other section of , 
the ha.ndbook would contain multiple descriptors for 
reference , . 

9,. Rationale: The handbook '|[hould bring the user to "an 
increased level of consciousness," This wilT. help 
him (or her) speed up the decision-making process for . . 
more efficient use of the handbook in the future;. % 

10, The situational problems should contain "classic" _ 
kinds of problems frequently encountered in the . 
implementation of products, (Program staff should draw 
on past research. activities and field application 
records for these experiences,) ' - ' 

1\, It is anticipated tjie handbook will be used both as a 
guide for planning future strategies ' and a tool^ for 
analyzing past mistakes and successes . This suggests ^ 
the need for carefu^-ly designed*- workslateferts which will^ 
leave a clear record of acrtions^ taken, . ^ V ^ 

12, Each alternative tactic or strategy should be associated 
with a discission of the enabj-ing resources required for 

. the tactic to be, irjpleinented , * 

' '. ■ - ' * ■ . ^ " ■ . 

13, The *di^cussion of tactic consequences could be stated 
as "iff then, hypotheses- " 

14, Display materiaj-s in the handbook by (1) keeping the model 
in front of people, (2) coding the model to. sections 

of the handbook, and ( 3) keeping a process orientation. 

General conscience was achieved among particif>ants (Budke , 
Howard, Hull -and- Kestem) in the day ' s discussion on the 
following major sections of the handbook: 



•I. Introduction ^ 

- An overview of the handbook 

- A discussion of the model t 

II • '^Planning a Strategy V^';:^ 
-t ^ • • . * ' " ■■ ■ . ' 

— This is the heart of the document, :=tontainirig check- 
lists and descriptions of innovation characteristics, 
^ .client situations, advocate position" descriptions > 
and .'installatioia tactics. 

m. Implementation of the Strategy 

- This section of i;he handbook will allow the user to 
concretize" tlie strategy with date^, identification 
of individuals, and other events which will" allow 

- tangible , asse'bsment of pmogress.* 

- A record of progress and unanticipated problems will 
be- contained ifi this section. 

IV. Assessing the, Impact of the Strategy 

- -■ ■ • ■ . ' ' 

- The questions and checklists in this section will 
be designed to answer th^ question of why were the 
tactics effective or not ^effective . 

- Intervening ^variables not an'ticipated will be noted 
(from the. Records section of ' the . handbook) . 

ApB^endices ' , r 

A. A* Typology of Situational Problems 

- This section would index constraints in situations, 
which would need to be taken into account in the ^ 
formulation of a diffusion strategy. ^ 

- Ratheif 'prescriptive alternative tactics would need 
> ^ t5 be listfed by each constraint. 

* * .. . c '" 

'■-..*'■ ■ ,Q • 

B. Careeri-'Education Dis^cussion 

■ - , without' defining the concept discrete terms, this 
section should discuss those characteristics which 
^ call : for : unique consideration in the implementation^ 

of^^odiicts- * . 

^ C. Reference, for Assessment Tools > 
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\ 2., SUMMARY f HE July 8, 1974 MEETING 

' WITH THE REVISION COMMITTEE^i^ 

Committee members were enthusiastic about the prospect of 
a career education -^implementation' handbook being develoj^d. 
They cited instaDces where such a handbook would be useful, ' 
e/g- helping /project directors in Florida conform to. a^ legal 
mandate to install career education in public sjchools by June. 
1977,' assisting seqond generation project direcStors in designing 
a pJ^ogram of career education, etc. One s^ate represented is" 
developing a project director's handbook. / 

1 • Target Audience . The coinmittee members reminded the 
developers that the responsibility for implementing and- in- 
stalling career education materials rests with - local personnel . 
State level staff can provide an array of possible options, 
to local staff for examination and awareness bvit &tate persons 
do no t r ecommenia or adyocate products.' -Personnel in State 
Education Agencies should be used to disseminate and es^lain 
th^ /handbook to local pro ject v planners ; btate directors have 
a* si^riificant role provii3ing**%^c*^^ to local 

pirojed-t direCtor;s. Local perigj^^i must' select and implement 
the materials; it is this audii^^e which should be addressed 
by -the handbook. It was the bpTnion of the revision committee \ 
that the handbook would be mo s,t' useful to second generation 
project directors: those individuals who were just starting^ 
caJ^eei: education programs. , • • 

The target audience should not be identified by title. 
For example, many of the "project directors in local education 
agi^ricies have many responsibilities. This, title/ project 
director, contains many different connotations depending on 
the size of tnl LEA and other extenuating circum§,tances . The 
cominittee recommended a functional title like^rpject planner or 
facilitator be. giVen. to the target audience. -Th^si would allow 
many different pemsons -in the organizational hierarqhy to 
identify with the mission of^ the handbook. ' . . 

2. Scope of Work , Clearly, the local > pro ject directors \ 
on ^ the revision committee^f elt a need for. inf ormai:ion' on how to 
initiate and Organize a project. This "felt need" extended 
into the arena of product development activities. The handbook 
may"^have -to address this concern (in the introduction section) 
ftom the viewpoint of the installation strategy £o be devised; 
butf the hapdbbok should limit its primary focus to the 
installation of career education^ products . 
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3. Organization of the Handbook / A rationale for each 
section should precede the checklist. The rationale should 
indicate-why it is important to consider the variables^listed 
iiv the checklist when formulating an installatidn straBejy. ^ 
The committee endorstiCl tlte-^hecklist as a means of- providing T 
a" quick overview of variables in the major domain. The nar- 
rative following the checklist should include a discussion of 
each variable. * ^ 

The index should include labels which project directors and 
other "f^ilitators" can relate to such as Inservice' Education 
of Teachers, Evaluation, the use of Advisory Committees, etc." 

'4.1 Content of the Handbook . The handbook should' contain 
' a section which allows. the user to note important antecedent 
conditions which would tend to facilitate or inhibit the 
installation of a career education product. 

^ > . ' . ..... 

4^2 The handbook should contain a section which suggests 
to the user sources of information for tasks not cover-ed , by the 
handbook. For example, ,the pser may be referred to the RUPS 
package, for use in *stating a problem situation. 

4.14- A section o^^ the handbook should address the organiza- 
tion of„ the staff. Perhaps ways of organizing staff could be 
suggested as illustrations- of ways of establishing credibility, 
efficiency, etc. The. use of diff erentiated ' staff ing could be 
suggested . 

4.15 The need for commitment from the Superintendent ^ould 
be«» stressed . His or her involvefteiit should bei^mi ted to 
policy matters and legal cor/cerns ^ Tactics s^^fc5a^ be. suggested 
which will communicate the ^uthor^H^flf his- of jEice. ^ 

4 .16 The manual st^s^uld be limfHid to*^ th.e' installation ^ 
of- products only^ 

4.17 Emphasize adaptajtion rather than adopt^or>. of mateiria'Is, 

4.18^ The checklistSvjp^y relate to ^.kntecedent conditions as 
a tool in diagnosing situations for prescribing tactics. 

4.19 The tactics should ^^.o vide several opticus fpr local^ 
project directors. '^ . ' ^* ■ » ' - , 

■ : ,.y S ■ r-rj ■■■■ • ' . ■ ^ - ■ , ■ . 

4.20 The handbook ° sFiould ",Jiave a rs€rongly r ational tone' .' 

^ " 



It should be wr.ittjeA'in cphversational style.- 
A. 22 The publication sbcMld beJ dperi, simple; brief,, 



consider using cartoons', etc. 
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4.23' The handbook should assume the readers will have some 
jram planning expei 
be listed a^ a reminde 



program planning expeBtence; ne verthele ss , the elements should 

id^, 



4.24 An unintended outcome _o£_iiie_ handbook may be its 
use (to researchers) as a tool f5f~~ana iy z i i^g the process of 
installing- products in^field sii:^T5 '( including the comparison 
of R&D studies) . 

4.25. Handbook users need lists of alternatives ( ib the 
tactic selection section) for jAealing with the situation (resolving 
the perceived problem) . • ~ ' , , 

4.26 The index must be clear, easy to use.'., 

4 . 27 "Pet" terms must be eliminated . 

4.28 Theoretical concepts must be scaled down to common 
language . - ' . 

4.29 ' The overall mission of th,e handbook is "how to do it," 
but users must understand reasons why the use of certain concepts 
are important, in order to select the appropriate tactic. 

4 . 3D The cookbook format is alright. 



5.0 Terminology . The revision cOT&nittee indicated the 
terms "produc.t" and "advocate" had negative connotations- 
Also the handbook should include a glossary of terms. • 

S^. Suggestions for Revision in the Evaluation Instruments 

6..1 Include a tear sheet in variouh/ locations throughout 
the handbook' and request it be mailed to the product developers 
This worksheet could dearl with incidents which are influencing 
installation activities. i 

6.2 The stage of development of the project should be • 
included on the backgroun^d information questionnaire. 

6.3 The "users of the product" question should be written 
in more of an open-ended fashion. 

6.4 The "Critical Incidents Report" should be incorporated, 
into the "implementations section" of the handbook. (/The 
incidents should be positive as well as. negative. , 

_.6.5 The Formative Evaluation Questions should be 'limited 
to 5 or 6 which can be asked ^for each ma jor^ section of the 
Handbook. For the most part, summative types of questions 
should be. left to the third party evaluation. < ^ . 
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' Minutes of the CEPIH Program Staff Meeting 7/10/T4; 
Reactions to the Revision ^Committee Recommendations 



Persons Present; John Howard, Bill ^ Hull , Ralph Ke^ter 

In general, tjie program staff agreed with most of the / 
recommendations from the Revision Committee. However , the fol- 
lowing excS^tions should be noted: 

1 . The negative connotation for the VordS. "product" ^• 
and "advocate, " We have beqji using these terms along with Vthe 
term "installation" to denote a research and development f:name 
of reference. We are certain that many local project staff 
personig; prefer to talk about "program" rather than '^product," 
And th^ prefer ±o dj-scuss "adaptation" rajbher than "adoption." 
This tendency -may be'.'^^ to the scarcity of. bona fide research 
and development pfodlrots which are available for installation. 
Such a scarcity is regretable but we have a responsibility, 
to promote terminology which. is consistent with our way of 
thinking. The terms "ag^ocate" and "product" have a very 
technical meaning as theA are used in the program. Our judgment 
is to continue'^to u^e succ terms . (as well as "installation") 
in the development of, the handbook. The t^ms should be defined 
early 'in the handbook and references to" these definitions may 
be included in later sections of the publication. T 

Interpretation of How to Do It Handbook . We haive the . 
that^the local care<»r education project directors may 
ifferent perception of the content for a ^'how to do it" 
ook than^ the scope of work- delineated for the CEPIH. 
committee members want information on how to start a program 
and 'substantive- information such as how to identify busi^ss^s 
as potential training centers. They do not seem to perceive 
th^ ^ndbook as a tool for analyzing past mistakes' and re- 
for^lating installation tactics. We remain convinced bf the 
lner^« of a process approach to-^roduct installation. Illustra- 
tions of tMe substance of cateeif education^will be included^ 
in the handbook at appropriate places but the handbook will^ 
f ocusS-^^riinarily on the process of installing products. * 
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*. _ - . S: '6/26/74 

• 3. SUGGESTED GUIDELINES FOR NOMINATING 
LOCAL EDUCATION AGENCIES 

" ^ . ■ - . . 

!• The primary consideration in the nomination oH^school 
distjricts to participate »in tljis program t:o develop a Career * 
Education Product Installation Handbook is the availability of 
career educfa,tion products to be installed on site during the 
1974-75 . school year A "product", is defined as a curriculum 
. unit or guide which relates students to careers or a manual ' ' 
which capacitates an organization or agen6y to provide career*- 
education activities J In. any event , the product should 
have a tangible outcome, something which can be observed. The' 
product should Rave a 'relatively single focus (for example, 
the establishment of an office within the school to assist 
students in finding employment in business and industry); 
but the product should require its use for a specific period of 
time for example, the teaching of ^n instructional un4t. 
The product should be ready for installation during September 
1974 / Th^ product may be developed in-house, within the local ^ 
school district, or transported to the district from- a developing 
agency outside of the local education agency. _ " ' 



5. Most of the career, education project directors should 

be from urban and suburban locations but at least one rural site 
shpuld be represented. ^ . 

■ • ■ , - ; ' . ' ■ ■ . ■' \ ' ^ . . 

6. Ethnic groups shblild be represented among the sites 
nominated from each state whenever possible. 



2. The career education -project, director in the nominated 
district should desire to participate in the program. His or 
her participation should be volurrtary. Program demands, upon / 
the project directprs will be minimal : two or three telephone 
conver.sations plus a questionnaire at the end of the six months . 
use of the handbook*. An orient;atioh meeting of the project . 
directors is planned. ;in at -least one state. * . ^ ' 

3. . At least one female project director should be included 
among the sites nominated . * * 

4. At least one regional service* centner should be included . 
in the list of sites if this fits the organizational pattern. 

of the state. 
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Wa realize it will be di'f£i<:uit to identi-fy valid and" ^ 
reliable products which are ready for use in a local school 
district setting . ' The task becomes one. of selecting sites 
with, the best available career education products. Feel 
. f re^e to phone Wm. Hull in Columb|Lis, Ohio (614) 486-3655 if you 
_ha^ ye q uestions or need addition^, information. . ^ _ _ 



APPENDIX D 

F*0RMATIVE EVALUATION SETTINGS. AND RESULTS 
1. Characteristics of the Field Site Settings ' 
a.- Number of Schools Using Handbook by Grade 
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b. Types of School Districts by States 







\. 






Urban 


. Suburban or \ 
Moderate Size \ 


Small Rural 
Sites 


TEXAS 


Ft. Worth 
Dallas 


Sulphur Springs \ 
McKin^ey ' \ 

kJ W C C W C4 J_ \ 


Honey Grove 
Harlingen 


FLORIDA 


Orlando Co. 
Pinellas Co. 
Broward Co. 


Alachua Co. \ 
Sarasota Co. \ 
Leon Co. \ 


Wakulla Co, 

Brevard 

Okaloosa^ 


. OHIO 


Lorain 


Mansfield | 


Mad-River 
Green 

y ' 



J. . ■ , ■ r ■ I 

unique setting, Spanish speaking people 

2 . • 

unique setting, air force base within the county 
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■2. - Res'ults pf the Formatiye Evaluation 

' . ■ " IK " . ■ ■ -'^ ; ■ ■ 

Name ^ 



r 



- FORMATIVE ^EVALUATION QUESTIONNAIRE 
/Career Education Product Installation Handbook' (CEPIH) • 



The CEPIH Formative Evaluation. Questionnaire is design^, to- 
%ather inf orination f or revising amd improving the Handb'ook. Your 
comments and suggestions / in/ combination 'with>those of others who 
are' field Resting the Handbook,, will be used to^ revise the Hand- 
book prior .to its final evaluation aitd testing. c '■ ' ' 

r : ' r ' .\ ' - ^ ^ v^. • - : 

The information you provide willtnot be as^o*iciated with, your 
name in reporting' the data. ^ Your ^response will be used ' in the 
analysis of ^roup data only. We appreciate • and val.ue^your pro- 
fessional judgement -and suggestions. Participation^^ln this activ- 
ity is voluntary.' ' . ^ ' • 

, \ There are three sections to this questionnaire. ' The first , 
section concer^is your percep-J^on of potential target audiences 
for, CEPIH ^and appears on this pagei The second section concerns 
your evaluation of each majojr seption of CEPIH and appears on the 
following pages. The third" sectidh provides documentary infa»rma- 
tion will be kept completely confidential. Please read the 

^directions for each question. anc3 attempt to ^ake your responses ' 
as accurate as possible. - 

Section 1 , * ; . 

This list represents specific groups of people that could poten- 
tially benefit from receiving CEPIH. Please cheek (7) ^ four (4) 
groups that you think could benefit most by ' receiving CEPIH. 

1. Classroom Teaghers 

■ . . . < . * ■■ ■ 

// 2. Curriculum D.evelopers ' ' 

r . ■ . • ■ ' , 

' 3 3'. Graduate Teacher Education. Pers8nriel 

' ' ^Z^^-T^* Local Supervisors 

3 . 5. Researchers ^ ■ ' ^ 

/ ^ ' 6 . School Administrators ' ■ ' 



?• state Departrnent Administrators 
8*. ..Stat^' Department Consultants 
S t ^ t ^ D e p ar trj e n t S \>p er vi s o r s 



9. 

_10. 

' 12, -pther (specify) 



Teq^Oj^r Education Chairmen - . 

iDndfergraduate Teacher Education Personnel 



Section II 



To complete this questionv you, simply have tov(l) check (VJ^if YOU 
STRONGLY AGREE, AGREE ^ HAVE NO OPINION , DISAGREE , OR STRONGLY 
DISAGREE with .each/of the six statements lor e^ch of the four.-; 
sections lof the Handbook; and (2) add a shor£ explanatiori of the 
problem or, suggested change related to /tiie appropriate statement y 
and section. - , « ' * <a , 
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' ' 1- , . . . . ■ 

, • ^ OTLANATION OF PROBLEMS JoR 
■SUGGESTED CHANGES ^ ' 

/■ ■ ■: . ' ■ ■ 


1. , .The ihateriai . included 
in; each of the following ' 
s'ect'ions waSvUsef ul: 

, av -liitroductioij 


■ ' ' , ' ^ ■ . '< 

, SA A ' D SD 


' /' ■ ; ' ' ''^^ 


.. b. Planning : ' 
. c.^ 'Implementation *^ • ' 


SA A N i) 'SD 
SA ' A N D SD 


./ • ■ " ' ; /• ; c-m 

*' ' ' " / j ' ' ■ . ■, ' ■ . 


Q. Assessing 1 • , , 

• ■ ' 1 ' . ■ ■ ' ■ ■ ' ,, 
If ' ' 


■ 1 ■ * 

SA/ A N ^ p 'SD 


■ ' , ■■ ■ 


■ . i ■ . ■ ^. ■■ ■ , , 

t ^ ■ . ■ i ' I ■'. ' ' '* , . 

2 . , The iinfonnation in each - 
of the following sections' 
was easy lo find: ' , 

a/ Intro^duction 


' ■, ; " ." ■ ■ ' ■ ' 
■ ' . " ■ * 

11 . , 1 ' 

SA A H D SD 


* ■ . ' ■ ■ /'", \ 

' : .'■■vf '-.. - 


■ b. ; Planning 


SA A N D ,SD 

* /y y ' \ 


■: ' • • A ' ■ " .■■ ■ 


/ ' c. 'implenientation 

. i . ■ ' . 


i . 

SA A N-.'D SD 
77 7 ^ 


; . ■ 

; 

1 ^ J ■■ . .• . 


.? ■• . ( • . .. 
d. Assessing 

CD 1/^ 1 J 


SA A N D ' SD , 

4 15 / - 


, ■ ' ". * ■ 

1 .J - ■ 

.99 ■ %■ ; . - ^ ' ' ' 
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Serious biases (^.g.) 
cultuijal, sexual, 
economidj.etc.) have 
been elisiinated from 
the materials in each 
of the following 
sectibhs: 



a. Introduction ^ 



SA A N D SB 



■1 . 



b. Planning 



SA, A, « D SD 

0 lC_ 



c. 'implementation 



'A. N. ,D SD 



5^ -A ^ N, 



d« Assessing 



SA A N D SD 



The terminology used 
in each of the following 
' sections was easily > 
understood: 

1 1, , II ^ ' 

a. Introduction 



b.' Planning 



a*n;d SD, 



5A a n 'D SD 



c. Implementation 



SA . A^ N D,, SD 



> 

4 



d.; Assessing * 



SA A \ ' D SD 



5 . The organization "of the 

^ • ,m ^6ach of the 

'following sections was 
■ , meaningful ; ' 



a, introduction |. 



^SA A N ,D SD 



7 - // 




b. Planning 



SA 'A N \ SD 

7 )o ' r ' 



{ c, Implementation ' 



SA A N D 



SD 



d. / As'$essing 



n 



SA- A./ N 



D SD. 



,6. the o.i;ganization of 

each of" the following , 
gectijOns encouraged 
active planning of 

installation strategies ; 

■ « • ' ■ 

a» Introduction 



SA 



N D :SD 



^_ K J^ J. 



b. Planning, 



SA .A N D SD " 
A X - 



c, ' Implementation 

_ V _ _ . 

d. ' Assessing' 



SA A " N 
SA A,. N 

^, /i / 



D SD 

D'' SD 
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